unleashed-firmware/SConstruct

379 lines
9.6 KiB
Python
Raw Normal View History

#
# Main Flipper Build System entry point
#
# This file is evaluated by scons (the build system) every time fbt is invoked.
# Scons constructs all referenced environments & their targets' dependency
# trees on startup. So, to keep startup time as low as possible, we're hiding
# construction of certain targets behind command-line options.
import os
from fbt.util import path_as_posix
DefaultEnvironment(tools=[])
EnsurePythonVersion(3, 8)
# Progress(["OwO\r", "owo\r", "uwu\r", "owo\r"], interval=15)
# This environment is created only for loading options & validating file/dir existence
fbt_variables = SConscript("site_scons/commandline.scons")
cmd_environment = Environment(
toolpath=["#/scripts/fbt_tools"],
tools=[
("fbt_help", {"vars": fbt_variables}),
],
variables=fbt_variables,
)
# Building basic environment - tools, utility methods, cross-compilation
# settings, gcc flags for Cortex-M4, basic builders and more
coreenv = SConscript(
"site_scons/environ.scons",
exports={"VAR_ENV": cmd_environment},
toolpath=["#/scripts/fbt_tools"],
)
SConscript("site_scons/cc.scons", exports={"ENV": coreenv})
# Create a separate "dist" environment and add construction envs to it
distenv = coreenv.Clone(
tools=[
"fbt_dist",
"fbt_debugopts",
"openocd",
"blackmagic",
"jflash",
],
ENV=os.environ,
UPDATE_BUNDLE_DIR="dist/${DIST_DIR}/f${TARGET_HW}-update-${DIST_SUFFIX}",
Improve vscode clangd experience (#2431) * Improve vscode clangd experience - Resolve and use absolute path for the toolchain. This allows clangd to use compile_commands.json file without running under fbtenv, simplifies setup for vscode clangd extension. As a side effect, a rebuild is needed to update compile_commands.json after moving the source tree. - Add the recommended default settings of the extension to settings.json. * Use build/latest for compile-commands-dir This makes it behave closer to c-cpp-properties. * Reformat crosscc.py This is a PEP-8 violation but black seems to enforce it * Bypass --query-driver This has some security implications as it grants clangd the ability to execute any executables anywhere while trying to probe a compiler based on CDB. However it's very hard to do this the safe and intended way without resorting to config generation due to reason listed in #2431. Besides that we already have workspace trust so what could go wrong? ;) * Add an option for vscode_dist to switch between clangd and cpptools This will install different extensions.json tuned for either clangd or cpptools based on user selection. It will also install c_cpp_properties.json when using cpptools since clangd doesn't use this file. The root .gitignore now also doesn't accidentally ignore everything under the .vscode directory. * Use absolute path for .vscode gitignore Turns out the previously used "relative" paths aren't even valid gitignore patterns and to actually do what it means one needs to use the absolute paths instead. * Handle variable parsing in commandline.scons commandline.scons is the place where all other command line parsing happens. Move LANG_SERVER variable parsing there and add a constraint to make the code more consistent. --------- Co-authored-by: hedger <hedger@users.noreply.github.com> Co-authored-by: hedger <hedger@nanode.su>
2023-08-16 04:23:09 +03:00
VSCODE_LANG_SERVER=ARGUMENTS.get("LANG_SERVER", "cpptools"),
)
firmware_env = distenv.AddFwProject(
base_env=coreenv,
fw_type="firmware",
fw_env_key="FW_ENV",
)
# If enabled, initialize updater-related targets
if GetOption("fullenv") or any(
filter(lambda target: "updater" in target or "flash_usb" in target, BUILD_TARGETS)
):
updater_env = distenv.AddFwProject(
base_env=coreenv,
fw_type="updater",
fw_env_key="UPD_ENV",
)
# Target for self-update package
dist_basic_arguments = [
"--bundlever",
"${UPDATE_VERSION_STRING}",
]
dist_radio_arguments = [
"--radio",
"${ROOT_DIR.abspath}/${COPRO_STACK_BIN_DIR}/${COPRO_STACK_BIN}",
"--radiotype",
"${COPRO_STACK_TYPE}",
"${COPRO_DISCLAIMER}",
"--obdata",
"${ROOT_DIR.abspath}/${COPRO_OB_DATA}",
New clock switch schema, fixes random core2 crashes (#3008) * Updated stack to 1.17.0 * hal: ble: Fixed stack config * Bumped stack version in config * scripts: added validation of copro stack version in update bundles * Copro: update to 1.17.2 * FuriHal: adjust tick frequency for HSE as sys clk * FuriHal: adjust systick reload on sys clock change * Sync api and format sources * scripts: updated ob.data for newer stack * FuriHal: return core2 hse pll transition on deep sleep * FuriHal: cleanup ble glue * FuriHal: rework ble glue, allow shci_send in critical section * FuriHal: sync api symbols * FuriHal: cleanup BLE glue, remove unused garbage and duplicate declarations * FuriHal: BLE glue cleanup, 2nd iteration * FuriHal: hide tick drift reports under FURI_HAL_OS_DEBUG * Lib: sync stm32wb_copro with latest dev * FuriHal: ble-glue, slightly less editable device name and duplicate definition cleanup * FuriHal: update ble config options, enable some optimizations and ext adv * FuriHal: update clock switch method documentation * FuriHal: better SNBRSA bug workaround fix * FuriHal: complete comment about tick skew * FuriHal: proper condition in clock hsi2hse transition * FuriHal: move PLL start to hse2pll routine, fix lockup caused by core2 switching to HSE before us * FuriHal: explicit HSE start before switch * FuriHal: fix documentation and move flash latency change to later stage, remove duplicate LL_RCC_SetRFWKPClockSource call --------- Co-authored-by: hedger <hedger@nanode.su> Co-authored-by: hedger <hedger@users.noreply.github.com>
2023-09-19 17:22:21 +03:00
"--stackversion",
"${COPRO_CUBE_VERSION}",
]
dist_resource_arguments = [
"-r",
firmware_env.subst("${RESOURCES_ROOT}"),
]
dist_splash_arguments = (
[
"--splash",
distenv.subst("assets/slideshow/$UPDATE_SPLASH"),
]
if distenv["UPDATE_SPLASH"]
else []
)
selfupdate_dist = distenv.DistCommand(
"updater_package",
(distenv["DIST_DEPENDS"], firmware_env["FW_RESOURCES_MANIFEST"]),
DIST_EXTRA=[
*dist_basic_arguments,
*dist_radio_arguments,
*dist_resource_arguments,
*dist_splash_arguments,
],
)
selfupdate_min_dist = distenv.DistCommand(
"updater_minpackage",
distenv["DIST_DEPENDS"],
DIST_EXTRA=dist_basic_arguments,
)
# Updater debug
distenv.PhonyTarget(
"updater_debug",
"${GDBPYCOM}",
source=updater_env["FW_ELF"],
GDBREMOTE="${OPENOCD_GDB_PIPE}",
)
distenv.PhonyTarget(
"updater_blackmagic",
"${GDBPYCOM}",
source=updater_env["FW_ELF"],
GDBOPTS=distenv.subst("$GDBOPTS_BLACKMAGIC"),
GDBREMOTE="${BLACKMAGIC_ADDR}",
)
# Installation over USB & CLI
usb_update_package = distenv.AddUsbFlashTarget(
"#build/usbinstall.flag",
(firmware_env["FW_RESOURCES_MANIFEST"], selfupdate_dist),
)
distenv.Alias("flash_usb_full", usb_update_package)
usb_minupdate_package = distenv.AddUsbFlashTarget(
"#build/minusbinstall.flag", (selfupdate_min_dist,)
)
distenv.Alias("flash_usb", usb_minupdate_package)
# Target for copying & renaming binaries to dist folder
basic_dist = distenv.DistCommand("fw_dist", distenv["DIST_DEPENDS"])
distenv.Default(basic_dist)
[FL-3097] fbt, faploader: minimal app module implementation (#2420) * fbt, faploader: minimal app module implementation * faploader, libs: moved API hashtable core to flipper_application * example: compound api * lib: flipper_application: naming fixes, doxygen comments * fbt: changed `requires` manifest field behavior for app extensions * examples: refactored plugin apps; faploader: changed new API naming; fbt: changed PLUGIN app type meaning * loader: dropped support for debug apps & plugin menus * moved applications/plugins -> applications/external * Restored x bit on chiplist_convert.py * git: fixed free-dap submodule path * pvs: updated submodule paths * examples: example_advanced_plugins.c: removed potential memory leak on errors * examples: example_plugins: refined requires * fbt: not deploying app modules for debug/sample apps; extra validation for .PLUGIN-type apps * apps: removed cdefines for external apps * fbt: moved ext app path definition * fbt: reworked fap_dist handling; f18: synced api_symbols.csv * fbt: removed resources_paths for extapps * scripts: reworked storage * scripts: reworked runfap.py & selfupdate.py to use new api * wip: fal runner * fbt: moved file packaging into separate module * scripts: storage: fixes * scripts: storage: minor fixes for new api * fbt: changed internal artifact storage details for external apps * scripts: storage: additional fixes and better error reporting; examples: using APP_DATA_PATH() * fbt, scripts: reworked launch_app to deploy plugins; moved old runfap.py to distfap.py * fbt: extra check for plugins descriptors * fbt: additional checks in emitter * fbt: better info message on SDK rebuild * scripts: removed requirements.txt * loader: removed remnants of plugins & debug menus * post-review fixes
2023-03-14 17:29:28 +03:00
dist_dir_name = distenv.GetProjetDirName()
dist_dir = distenv.Dir(f"#/dist/{dist_dir_name}")
external_apps_artifacts = firmware_env["FW_EXTAPPS"]
external_app_list = external_apps_artifacts.application_map.values()
fap_dist = [
distenv.Install(
[FL-3097] fbt, faploader: minimal app module implementation (#2420) * fbt, faploader: minimal app module implementation * faploader, libs: moved API hashtable core to flipper_application * example: compound api * lib: flipper_application: naming fixes, doxygen comments * fbt: changed `requires` manifest field behavior for app extensions * examples: refactored plugin apps; faploader: changed new API naming; fbt: changed PLUGIN app type meaning * loader: dropped support for debug apps & plugin menus * moved applications/plugins -> applications/external * Restored x bit on chiplist_convert.py * git: fixed free-dap submodule path * pvs: updated submodule paths * examples: example_advanced_plugins.c: removed potential memory leak on errors * examples: example_plugins: refined requires * fbt: not deploying app modules for debug/sample apps; extra validation for .PLUGIN-type apps * apps: removed cdefines for external apps * fbt: moved ext app path definition * fbt: reworked fap_dist handling; f18: synced api_symbols.csv * fbt: removed resources_paths for extapps * scripts: reworked storage * scripts: reworked runfap.py & selfupdate.py to use new api * wip: fal runner * fbt: moved file packaging into separate module * scripts: storage: fixes * scripts: storage: minor fixes for new api * fbt: changed internal artifact storage details for external apps * scripts: storage: additional fixes and better error reporting; examples: using APP_DATA_PATH() * fbt, scripts: reworked launch_app to deploy plugins; moved old runfap.py to distfap.py * fbt: extra check for plugins descriptors * fbt: additional checks in emitter * fbt: better info message on SDK rebuild * scripts: removed requirements.txt * loader: removed remnants of plugins & debug menus * post-review fixes
2023-03-14 17:29:28 +03:00
dist_dir.Dir("debug_elf"),
list(app_artifact.debug for app_artifact in external_app_list),
),
*(
distenv.Install(
[FL-3097] fbt, faploader: minimal app module implementation (#2420) * fbt, faploader: minimal app module implementation * faploader, libs: moved API hashtable core to flipper_application * example: compound api * lib: flipper_application: naming fixes, doxygen comments * fbt: changed `requires` manifest field behavior for app extensions * examples: refactored plugin apps; faploader: changed new API naming; fbt: changed PLUGIN app type meaning * loader: dropped support for debug apps & plugin menus * moved applications/plugins -> applications/external * Restored x bit on chiplist_convert.py * git: fixed free-dap submodule path * pvs: updated submodule paths * examples: example_advanced_plugins.c: removed potential memory leak on errors * examples: example_plugins: refined requires * fbt: not deploying app modules for debug/sample apps; extra validation for .PLUGIN-type apps * apps: removed cdefines for external apps * fbt: moved ext app path definition * fbt: reworked fap_dist handling; f18: synced api_symbols.csv * fbt: removed resources_paths for extapps * scripts: reworked storage * scripts: reworked runfap.py & selfupdate.py to use new api * wip: fal runner * fbt: moved file packaging into separate module * scripts: storage: fixes * scripts: storage: minor fixes for new api * fbt: changed internal artifact storage details for external apps * scripts: storage: additional fixes and better error reporting; examples: using APP_DATA_PATH() * fbt, scripts: reworked launch_app to deploy plugins; moved old runfap.py to distfap.py * fbt: extra check for plugins descriptors * fbt: additional checks in emitter * fbt: better info message on SDK rebuild * scripts: removed requirements.txt * loader: removed remnants of plugins & debug menus * post-review fixes
2023-03-14 17:29:28 +03:00
dist_dir.File(dist_entry[1]).dir,
app_artifact.compact,
)
[FL-3097] fbt, faploader: minimal app module implementation (#2420) * fbt, faploader: minimal app module implementation * faploader, libs: moved API hashtable core to flipper_application * example: compound api * lib: flipper_application: naming fixes, doxygen comments * fbt: changed `requires` manifest field behavior for app extensions * examples: refactored plugin apps; faploader: changed new API naming; fbt: changed PLUGIN app type meaning * loader: dropped support for debug apps & plugin menus * moved applications/plugins -> applications/external * Restored x bit on chiplist_convert.py * git: fixed free-dap submodule path * pvs: updated submodule paths * examples: example_advanced_plugins.c: removed potential memory leak on errors * examples: example_plugins: refined requires * fbt: not deploying app modules for debug/sample apps; extra validation for .PLUGIN-type apps * apps: removed cdefines for external apps * fbt: moved ext app path definition * fbt: reworked fap_dist handling; f18: synced api_symbols.csv * fbt: removed resources_paths for extapps * scripts: reworked storage * scripts: reworked runfap.py & selfupdate.py to use new api * wip: fal runner * fbt: moved file packaging into separate module * scripts: storage: fixes * scripts: storage: minor fixes for new api * fbt: changed internal artifact storage details for external apps * scripts: storage: additional fixes and better error reporting; examples: using APP_DATA_PATH() * fbt, scripts: reworked launch_app to deploy plugins; moved old runfap.py to distfap.py * fbt: extra check for plugins descriptors * fbt: additional checks in emitter * fbt: better info message on SDK rebuild * scripts: removed requirements.txt * loader: removed remnants of plugins & debug menus * post-review fixes
2023-03-14 17:29:28 +03:00
for app_artifact in external_app_list
for dist_entry in app_artifact.dist_entries
),
]
Depends(
fap_dist,
[FL-3097] fbt, faploader: minimal app module implementation (#2420) * fbt, faploader: minimal app module implementation * faploader, libs: moved API hashtable core to flipper_application * example: compound api * lib: flipper_application: naming fixes, doxygen comments * fbt: changed `requires` manifest field behavior for app extensions * examples: refactored plugin apps; faploader: changed new API naming; fbt: changed PLUGIN app type meaning * loader: dropped support for debug apps & plugin menus * moved applications/plugins -> applications/external * Restored x bit on chiplist_convert.py * git: fixed free-dap submodule path * pvs: updated submodule paths * examples: example_advanced_plugins.c: removed potential memory leak on errors * examples: example_plugins: refined requires * fbt: not deploying app modules for debug/sample apps; extra validation for .PLUGIN-type apps * apps: removed cdefines for external apps * fbt: moved ext app path definition * fbt: reworked fap_dist handling; f18: synced api_symbols.csv * fbt: removed resources_paths for extapps * scripts: reworked storage * scripts: reworked runfap.py & selfupdate.py to use new api * wip: fal runner * fbt: moved file packaging into separate module * scripts: storage: fixes * scripts: storage: minor fixes for new api * fbt: changed internal artifact storage details for external apps * scripts: storage: additional fixes and better error reporting; examples: using APP_DATA_PATH() * fbt, scripts: reworked launch_app to deploy plugins; moved old runfap.py to distfap.py * fbt: extra check for plugins descriptors * fbt: additional checks in emitter * fbt: better info message on SDK rebuild * scripts: removed requirements.txt * loader: removed remnants of plugins & debug menus * post-review fixes
2023-03-14 17:29:28 +03:00
list(app_artifact.validator for app_artifact in external_app_list),
)
Alias("fap_dist", fap_dist)
# Copy all faps to device
fap_deploy = distenv.PhonyTarget(
"fap_deploy",
Action(
[
[
"${PYTHON3}",
"${FBT_SCRIPT_DIR}/storage.py",
"-p",
"${FLIP_PORT}",
"send",
"${SOURCE}",
"/ext/apps",
]
]
),
source=firmware_env.Dir(("${RESOURCES_ROOT}/apps")),
)
Depends(fap_deploy, firmware_env["FW_RESOURCES_MANIFEST"])
# Target for bundling core2 package for qFlipper
copro_dist = distenv.CoproBuilder(
"#/build/core2_firmware.tgz",
[],
)
distenv.AlwaysBuild(copro_dist)
distenv.Alias("copro_dist", copro_dist)
firmware_flash = distenv.AddFwFlashTarget(firmware_env)
distenv.Alias("flash", firmware_flash)
# To be implemented in fwflash.py
firmware_jflash = distenv.AddJFlashTarget(firmware_env)
distenv.Alias("jflash", firmware_jflash)
distenv.PhonyTarget(
"gdb_trace_all",
"$GDB $GDBOPTS $SOURCES $GDBFLASH",
source=firmware_env["FW_ELF"],
GDBOPTS="${GDBOPTS_BASE}",
GDBREMOTE="${OPENOCD_GDB_PIPE}",
GDBFLASH=[
"-ex",
"thread apply all bt",
"-ex",
"quit",
],
)
# Debugging firmware
firmware_debug = distenv.PhonyTarget(
"debug",
"${GDBPYCOM}",
source=firmware_env["FW_ELF"],
GDBOPTS="${GDBOPTS_BASE}",
GDBREMOTE="${OPENOCD_GDB_PIPE}",
FBT_FAP_DEBUG_ELF_ROOT=path_as_posix(firmware_env.subst("$FBT_FAP_DEBUG_ELF_ROOT")),
)
distenv.Depends(firmware_debug, firmware_flash)
distenv.PhonyTarget(
"blackmagic",
"${GDBPYCOM}",
source=firmware_env["FW_ELF"],
GDBOPTS="${GDBOPTS_BASE} ${GDBOPTS_BLACKMAGIC}",
GDBREMOTE="${BLACKMAGIC_ADDR}",
FBT_FAP_DEBUG_ELF_ROOT=path_as_posix(firmware_env.subst("$FBT_FAP_DEBUG_ELF_ROOT")),
)
# Debug alien elf
debug_other_opts = [
"-ex",
"source ${FBT_DEBUG_DIR}/PyCortexMDebug/PyCortexMDebug.py",
# "-ex",
# "source ${FBT_DEBUG_DIR}/FreeRTOS/FreeRTOS.py",
"-ex",
"source ${FBT_DEBUG_DIR}/flipperversion.py",
"-ex",
"fw-version",
]
distenv.PhonyTarget(
"debug_other",
"${GDBPYCOM}",
GDBOPTS="${GDBOPTS_BASE}",
GDBREMOTE="${OPENOCD_GDB_PIPE}",
GDBPYOPTS=debug_other_opts,
)
distenv.PhonyTarget(
"debug_other_blackmagic",
"${GDBPYCOM}",
GDBOPTS="${GDBOPTS_BASE} ${GDBOPTS_BLACKMAGIC}",
GDBREMOTE="${BLACKMAGIC_ADDR}",
GDBPYOPTS=debug_other_opts,
)
# Just start OpenOCD
distenv.PhonyTarget(
"openocd",
"${OPENOCDCOM}",
)
# Linter
distenv.PhonyTarget(
"lint",
[["${PYTHON3}", "${FBT_SCRIPT_DIR}/lint.py", "check", "${LINT_SOURCES}"]],
LINT_SOURCES=[n.srcnode() for n in firmware_env["LINT_SOURCES"]],
)
distenv.PhonyTarget(
"format",
[["${PYTHON3}", "${FBT_SCRIPT_DIR}/lint.py", "format", "${LINT_SOURCES}"]],
LINT_SOURCES=[n.srcnode() for n in firmware_env["LINT_SOURCES"]],
)
# PY_LINT_SOURCES contains recursively-built modules' SConscript files
# Here we add additional Python files residing in repo root
firmware_env.Append(
PY_LINT_SOURCES=[
# Py code folders
"site_scons",
"scripts",
"applications",
"applications_user",
"assets",
"targets",
# Extra files
"SConstruct",
"firmware.scons",
"fbt_options.py",
]
)
black_commandline = "@${PYTHON3} -m black ${PY_BLACK_ARGS} ${PY_LINT_SOURCES}"
black_base_args = [
"--include",
'"(\\.scons|\\.py|SConscript|SConstruct|\\.fam)$"',
]
distenv.PhonyTarget(
"lint_py",
black_commandline,
PY_BLACK_ARGS=[
"--check",
"--diff",
*black_base_args,
],
PY_LINT_SOURCES=firmware_env["PY_LINT_SOURCES"],
)
distenv.PhonyTarget(
"format_py",
black_commandline,
PY_BLACK_ARGS=black_base_args,
PY_LINT_SOURCES=firmware_env["PY_LINT_SOURCES"],
)
# Start Flipper CLI via PySerial's miniterm
distenv.PhonyTarget(
"cli", [["${PYTHON3}", "${FBT_SCRIPT_DIR}/serial_cli.py", "-p", "${FLIP_PORT}"]]
)
# Update WiFi devboard firmware
distenv.PhonyTarget(
"devboard_flash", [["${PYTHON3}", "${FBT_SCRIPT_DIR}/wifi_board.py"]]
)
# Find blackmagic probe
distenv.PhonyTarget(
"get_blackmagic",
"@echo $( ${BLACKMAGIC_ADDR} $)",
)
# Find STLink probe ids
distenv.PhonyTarget(
"get_stlink",
distenv.Action(
lambda **kw: distenv.GetDevices(),
None,
),
)
# Prepare vscode environment
Improve vscode clangd experience (#2431) * Improve vscode clangd experience - Resolve and use absolute path for the toolchain. This allows clangd to use compile_commands.json file without running under fbtenv, simplifies setup for vscode clangd extension. As a side effect, a rebuild is needed to update compile_commands.json after moving the source tree. - Add the recommended default settings of the extension to settings.json. * Use build/latest for compile-commands-dir This makes it behave closer to c-cpp-properties. * Reformat crosscc.py This is a PEP-8 violation but black seems to enforce it * Bypass --query-driver This has some security implications as it grants clangd the ability to execute any executables anywhere while trying to probe a compiler based on CDB. However it's very hard to do this the safe and intended way without resorting to config generation due to reason listed in #2431. Besides that we already have workspace trust so what could go wrong? ;) * Add an option for vscode_dist to switch between clangd and cpptools This will install different extensions.json tuned for either clangd or cpptools based on user selection. It will also install c_cpp_properties.json when using cpptools since clangd doesn't use this file. The root .gitignore now also doesn't accidentally ignore everything under the .vscode directory. * Use absolute path for .vscode gitignore Turns out the previously used "relative" paths aren't even valid gitignore patterns and to actually do what it means one needs to use the absolute paths instead. * Handle variable parsing in commandline.scons commandline.scons is the place where all other command line parsing happens. Move LANG_SERVER variable parsing there and add a constraint to make the code more consistent. --------- Co-authored-by: hedger <hedger@users.noreply.github.com> Co-authored-by: hedger <hedger@nanode.su>
2023-08-16 04:23:09 +03:00
VSCODE_LANG_SERVER = cmd_environment["LANG_SERVER"]
vscode_dist = distenv.Install(
"#.vscode",
[
distenv.Glob("#.vscode/example/*.json"),
distenv.Glob(f"#.vscode/example/{VSCODE_LANG_SERVER}/*.json"),
],
)
distenv.Precious(vscode_dist)
distenv.NoClean(vscode_dist)
distenv.Alias("vscode_dist", vscode_dist)
# Configure shell with build tools
distenv.PhonyTarget(
"env",
"@echo $( ${FBT_SCRIPT_DIR.abspath}/toolchain/fbtenv.sh $)",
)