unleashed-firmware/scripts/sconsdist.py

280 lines
9.5 KiB
Python
Raw Normal View History

[FL-2263] Flasher service & RAM exec (#1006) * WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar #2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com> Co-authored-by: あく <alleteam@gmail.com>
2022-04-13 23:50:25 +03:00
#!/usr/bin/env python3
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
import json
[FL-2263] Flasher service & RAM exec (#1006) * WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar #2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com> Co-authored-by: あく <alleteam@gmail.com>
2022-04-13 23:50:25 +03:00
import shutil
import tarfile
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
import zipfile
from os import makedirs, walk
from os.path import basename, exists, join, relpath
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
from ansi.color import fg
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
from flipper.app import App
from update import Main as UpdateMain
[FL-2263] Flasher service & RAM exec (#1006) * WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar #2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com> Co-authored-by: あく <alleteam@gmail.com>
2022-04-13 23:50:25 +03:00
class ProjectDir:
def __init__(self, project_dir):
self.dir = project_dir
parts = project_dir.split("-")
self.target = parts[0]
self.project = parts[1]
self.flavor = parts[2] if len(parts) > 2 else ""
[FL-2263] Flasher service & RAM exec (#1006) * WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar #2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com> Co-authored-by: あく <alleteam@gmail.com>
2022-04-13 23:50:25 +03:00
class Main(App):
DIST_FILE_PREFIX = "flipper-z-"
DIST_FOLDER_MAX_NAME_LENGTH = 80
[FL-2263] Flasher service & RAM exec (#1006) * WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar #2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com> Co-authored-by: あく <alleteam@gmail.com>
2022-04-13 23:50:25 +03:00
def init(self):
self.subparsers = self.parser.add_subparsers(help="sub-command help")
self.parser_copy = self.subparsers.add_parser(
"copy", help="Copy firmware binaries & metadata"
)
self.parser_copy.add_argument("-p", dest="project", nargs="+", required=True)
[FL-2263] Flasher service & RAM exec (#1006) * WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar #2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com> Co-authored-by: あく <alleteam@gmail.com>
2022-04-13 23:50:25 +03:00
self.parser_copy.add_argument("-s", dest="suffix", required=True)
self.parser_copy.add_argument("-r", dest="resources", required=False)
[FL-2263] Flasher service & RAM exec (#1006) * WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar #2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com> Co-authored-by: あく <alleteam@gmail.com>
2022-04-13 23:50:25 +03:00
self.parser_copy.add_argument(
"--bundlever",
dest="version",
help="If set, bundle update package for self-update",
required=False,
)
self.parser_copy.add_argument(
"--noclean",
dest="noclean",
action="store_true",
help="Don't clean output directory",
required=False,
)
self.parser_copy.set_defaults(func=self.copy)
def get_project_file_name(self, project: ProjectDir, filetype: str) -> str:
# Temporary fix
project_name = project.project
if project_name == "firmware" and filetype != "elf":
project_name = "full"
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
dist_target_path = self.get_dist_file_name(project_name, filetype)
self.note_dist_component(
project_name, filetype, self.get_dist_path(dist_target_path)
)
return dist_target_path
def note_dist_component(self, component: str, extension: str, srcpath: str) -> None:
self._dist_components[f"{component}.{extension}"] = srcpath
[FL-2263] Flasher service & RAM exec (#1006) * WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar #2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com> Co-authored-by: あく <alleteam@gmail.com>
2022-04-13 23:50:25 +03:00
def get_dist_file_name(self, dist_artifact_type: str, filetype: str) -> str:
return f"{self.DIST_FILE_PREFIX}{self.target}-{dist_artifact_type}-{self.args.suffix}.{filetype}"
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
def get_dist_path(self, filename: str) -> str:
[FL-2263] Flasher service & RAM exec (#1006) * WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar #2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com> Co-authored-by: あく <alleteam@gmail.com>
2022-04-13 23:50:25 +03:00
return join(self.output_dir_path, filename)
def copy_single_project(self, project: ProjectDir) -> None:
obj_directory = join("build", project.dir)
[FL-2263] Flasher service & RAM exec (#1006) * WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar #2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com> Co-authored-by: あく <alleteam@gmail.com>
2022-04-13 23:50:25 +03:00
for filetype in ("elf", "bin", "dfu", "json"):
if exists(src_file := join(obj_directory, f"{project.project}.{filetype}")):
shutil.copyfile(
src_file,
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
self.get_dist_path(self.get_project_file_name(project, filetype)),
)
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
for foldertype in ("sdk_headers", "lib"):
if exists(sdk_folder := join(obj_directory, foldertype)):
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
self.note_dist_component(foldertype, "dir", sdk_folder)
# TODO: remove this after everyone migrates to new uFBT
self.create_zip_stub("lib")
def create_zip_stub(self, foldertype):
with zipfile.ZipFile(
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
self.get_dist_path(self.get_dist_file_name(foldertype, "zip")),
"w",
zipfile.ZIP_DEFLATED,
) as _:
pass
def copy(self) -> int:
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
self._dist_components: dict[str, str] = dict()
self.projects: dict[str, ProjectDir] = dict(
map(
lambda pd: (pd.project, pd),
map(ProjectDir, self.args.project),
)
)
project_targets = set(map(lambda p: p.target, self.projects.values()))
if len(project_targets) != 1:
self.logger.error(f"Cannot mix targets {project_targets}!")
return 1
self.target = project_targets.pop()
project_flavors = set(map(lambda p: p.flavor, self.projects.values()))
if len(project_flavors) != 1:
self.logger.error(f"Cannot mix flavors {project_flavors}!")
return 2
self.flavor = project_flavors.pop()
dist_dir_components = [self.target]
if self.flavor:
dist_dir_components.append(self.flavor)
self.output_dir_path = join("dist", "-".join(dist_dir_components))
[FL-2263] Flasher service & RAM exec (#1006) * WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar #2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com> Co-authored-by: あく <alleteam@gmail.com>
2022-04-13 23:50:25 +03:00
if exists(self.output_dir_path) and not self.args.noclean:
try:
shutil.rmtree(self.output_dir_path)
except Exception as ex:
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
self.logger.warn(f"Failed to clean output directory: {ex}")
[FL-2263] Flasher service & RAM exec (#1006) * WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar #2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com> Co-authored-by: あく <alleteam@gmail.com>
2022-04-13 23:50:25 +03:00
if not exists(self.output_dir_path):
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
self.logger.debug(f"Creating output directory {self.output_dir_path}")
[FL-2263] Flasher service & RAM exec (#1006) * WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar #2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com> Co-authored-by: あく <alleteam@gmail.com>
2022-04-13 23:50:25 +03:00
makedirs(self.output_dir_path)
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
for folder in ("debug", "scripts"):
if exists(folder):
self.note_dist_component(folder, "dir", folder)
for project in self.projects.values():
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
self.logger.debug(f"Copying {project.project} for {project.target}")
[FL-2263] Flasher service & RAM exec (#1006) * WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar #2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com> Co-authored-by: あく <alleteam@gmail.com>
2022-04-13 23:50:25 +03:00
self.copy_single_project(project)
self.logger.info(
fg.boldgreen(
f"Firmware binaries can be found at:\n\t{self.output_dir_path}"
)
[FL-2263] Flasher service & RAM exec (#1006) * WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar #2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com> Co-authored-by: あく <alleteam@gmail.com>
2022-04-13 23:50:25 +03:00
)
[FL-2263] Flasher service & RAM exec (#1006) * WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar #2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com> Co-authored-by: あく <alleteam@gmail.com>
2022-04-13 23:50:25 +03:00
if self.args.version:
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
if bundle_result := self.bundle_update_package():
return bundle_result
required_components = ("firmware.elf", "full.bin", "update.dir")
if all(
map(
lambda c: c in self._dist_components,
required_components,
)
):
self.bundle_sdk()
return 0
def bundle_sdk(self):
self.logger.info("Bundling SDK")
components_paths = dict()
sdk_components_keys = (
"full.bin",
"firmware.elf",
"update.dir",
"sdk_headers.dir",
"lib.dir",
"scripts.dir",
)
with zipfile.ZipFile(
self.get_dist_path(self.get_dist_file_name("sdk", "zip")),
"w",
zipfile.ZIP_DEFLATED,
) as zf:
for component_key in sdk_components_keys:
component_path = self._dist_components.get(component_key)
if component_key.endswith(".dir"):
components_paths[component_key] = basename(component_path)
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
for root, dirnames, files in walk(component_path):
if "__pycache__" in dirnames:
dirnames.remove("__pycache__")
for file in files:
zf.write(
join(root, file),
join(
components_paths[component_key],
relpath(
join(root, file),
component_path,
),
),
)
else:
# We use fixed names for files to avoid having to regenerate VSCode project
components_paths[component_key] = component_key
zf.write(component_path, component_key)
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
zf.writestr(
"components.json",
json.dumps(
{
"meta": {
"hw_target": self.target,
"flavor": self.flavor,
"version": self.args.version,
},
"components": components_paths,
}
),
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
)
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
def bundle_update_package(self):
self.logger.debug(
f"Generating update bundle with version {self.args.version} for {self.target}"
)
bundle_dir_name = f"{self.target}-update-{self.args.suffix}"[
: self.DIST_FOLDER_MAX_NAME_LENGTH
]
bundle_dir = self.get_dist_path(bundle_dir_name)
bundle_args = [
"generate",
"-d",
bundle_dir,
"-v",
self.args.version,
"-t",
self.target,
"--dfu",
self.get_dist_path(
self.get_project_file_name(self.projects["firmware"], "dfu")
),
"--stage",
self.get_dist_path(
self.get_project_file_name(self.projects["updater"], "bin")
),
]
if self.args.resources:
bundle_args.extend(
(
"-r",
self.args.resources,
)
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
)
bundle_args.extend(self.other_args)
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
if (bundle_result := UpdateMain(no_exit=True)(bundle_args)) == 0:
self.note_dist_component("update", "dir", bundle_dir)
self.logger.info(
fg.boldgreen(
f"Use this directory to self-update your Flipper:\n\t{bundle_dir}"
)
)
[FL-2263] Flasher service & RAM exec (#1006) * WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar #2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com> Co-authored-by: あく <alleteam@gmail.com>
2022-04-13 23:50:25 +03:00
[FL-3162] Moved ufbt to fbt codebase (#2520) * scripts: moved ufbt code * ufbt: fixed tool path * ufbt: fixed linter/formatter target descriptions * scripts: ufbt: cleanup * fbt: moved fap launch target to tools; ufbt fixes * fbt: fixed missing headers from SDK * ufbt: removed debug output * ufbt: moved project template to main codebase * ufbt: fixed vscode_dist * ufbt: path naming changes * fbt: error message for older ufbt versions * ufbt: docs fixes * ufbt: fixed build dir location * fbt: fixes for extapps objcopy * fbt: extapps: removed extra debug output; fixed formatting * ufbt: handle launch target for multiple known apps * ufbt: dropping wrapper; linter fixes * ufbt: fixed boostrap path * ufbt: renamed entrypoint * ufbt: updated vscode config * ufbt: moved sconsign db location * ufbt: fixed sconsign path * fbt: SDK builders rework * fbt: reworked sdk packaging * ufbt: additional checks and state processing * ufbt: fixed sdk state file location * dist: not packaging pycache * dump commit json content * Github: more workflow debug prints * Github: fix incorrect commit meta extraction in get_env.py * ufbt, fbt: changed SConsEnvironmentError->StopError * fbtenv: no longer needs SCRIPT_PATH pre-set * ufbt: fixed sdk state check * scripts: exception fixes for storage.py * scripts: fbtenv: added FBT_TOOLCHAIN_PATH for on Windows for compat * ufbt: app template: creating .gitkeep for images folder * ufbt: app template: fixed .gitkeep creation * docs: formatting fixes for AppManifests; added link to ufbt * fbt: added link to PyPI for old ufbt versions * sdk: fixed dir component paths Co-authored-by: Aleksandr Kutuzov <alleteam@gmail.com>
2023-04-06 05:44:37 +03:00
# Create tgz archive
with tarfile.open(
join(
self.output_dir_path,
bundle_tgz := f"{self.DIST_FILE_PREFIX}{bundle_dir_name}.tgz",
),
"w:gz",
compresslevel=9,
format=tarfile.USTAR_FORMAT,
) as tar:
self.note_dist_component(
"update", "tgz", self.get_dist_path(bundle_tgz)
)
tar.add(bundle_dir, arcname=bundle_dir_name)
return bundle_result
[FL-2263] Flasher service & RAM exec (#1006) * WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar #2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com> Co-authored-by: あく <alleteam@gmail.com>
2022-04-13 23:50:25 +03:00
if __name__ == "__main__":
Main()()