unleashed-firmware/scripts/flipper/utils/fff.py

107 lines
3.1 KiB
Python
Raw Normal View History

[FL-2183] [FL-2209] Dolphin Deeds, Level up, assets generation, refactoring (#965) * Desktop: cleanup headers * Get loader pubsub via record * [FL-2183] Dolphin refactoring 2022.01 * Restruct animations assets structure * Rename assets * Cleanup headers * Update Recording animation * Add BadBattery animation * Provide loader's pubsub via record * Fix load/unload animations * Scripts: add flipper format support, initial dolphin packager rework. Assets: internal and external dolphin. * Sync internal meta.txt and manifest.txt * Reorder, rename dolphin assets * Split essential generated assets * Add ReadMe for dolphin assets * Separate essential blocking animations * Scripts: full dolphin validation before packaging * Assets, Scripts: dolphin external resources packer * Github: update codeowners * Scripts: proper slots handling in dolphin animation meta * Scripts: correct frames enumeration and fix compiled assets. * [FL-2209] Add Dolphin Deeds points and many more * Remove excess frame_rate * Change dolphin assets directory * Scripts: add internal resource support to dolphin compiler * Scripts: add internal assets generation, renaming * Scripts: correct assert, renaming * Code cleanup, documentation, fixes * Update Levelup animations * Rename essential -> blocking * Fix Unlocked hint * Scripts: rewrite Templite compiller, replace regexps with token parser, split block types into code and variable blocks. Update dolphin templates. * Documentation: add key combos description and use information * Scripts: cleanup templit, more debug info and add dev comment Co-authored-by: あく <alleteam@gmail.com>
2022-01-29 12:20:41 +03:00
import logging
class FlipperFormatFile:
def __init__(self):
# Storage
self.lines = []
self.cursor = 0
# Logger
self.logger = logging.getLogger("FlipperFormatFile")
def _resetCursor(self):
self.cursor = 0
def nextLine(self):
line = None
while self.cursor < len(self.lines):
temp_line = self.lines[self.cursor].strip()
self.cursor += 1
if len(temp_line) > 0 and not temp_line.startswith("#"):
line = temp_line
break
if line is None:
raise EOFError()
return line
def readKeyValue(self):
line = self.nextLine()
data = line.split(":", 1)
if len(data) != 2:
self.logger.error(f"Incorrectly formated line {self.cursor}: `{line}`")
raise Exception("Unexpected line: not `key:value`")
return data[0].strip(), data[1].strip()
def readKey(self, key: str):
k, v = self.readKeyValue()
if k != key:
raise KeyError(f"Unexpected key {k} != {key}")
return v
def readKeyInt(self, key: str):
value = self.readKey(key)
return int(value) if value else None
def readKeyIntArray(self, key: str):
value = self.readKey(key)
return [int(i) for i in value.split(" ")] if value else None
def readKeyFloat(self, key: str):
value = self.readKey(key)
return float(value) if value else None
def writeLine(self, line: str):
self.lines.insert(self.cursor, line)
self.cursor += 1
def writeKey(self, key: str, value):
if isinstance(value, (str, int, float)):
pass
elif isinstance(value, (list, set)):
value = " ".join(map(str, value))
else:
raise Exception("Unknown value type")
self.writeLine(f"{key}: {value}")
def writeEmptyLine(self):
self.writeLine("")
def writeComment(self, text: str):
if text:
self.writeLine(f"# {text}")
else:
self.writeLine("#")
[FL-2183] [FL-2209] Dolphin Deeds, Level up, assets generation, refactoring (#965) * Desktop: cleanup headers * Get loader pubsub via record * [FL-2183] Dolphin refactoring 2022.01 * Restruct animations assets structure * Rename assets * Cleanup headers * Update Recording animation * Add BadBattery animation * Provide loader's pubsub via record * Fix load/unload animations * Scripts: add flipper format support, initial dolphin packager rework. Assets: internal and external dolphin. * Sync internal meta.txt and manifest.txt * Reorder, rename dolphin assets * Split essential generated assets * Add ReadMe for dolphin assets * Separate essential blocking animations * Scripts: full dolphin validation before packaging * Assets, Scripts: dolphin external resources packer * Github: update codeowners * Scripts: proper slots handling in dolphin animation meta * Scripts: correct frames enumeration and fix compiled assets. * [FL-2209] Add Dolphin Deeds points and many more * Remove excess frame_rate * Change dolphin assets directory * Scripts: add internal resource support to dolphin compiler * Scripts: add internal assets generation, renaming * Scripts: correct assert, renaming * Code cleanup, documentation, fixes * Update Levelup animations * Rename essential -> blocking * Fix Unlocked hint * Scripts: rewrite Templite compiller, replace regexps with token parser, split block types into code and variable blocks. Update dolphin templates. * Documentation: add key combos description and use information * Scripts: cleanup templit, more debug info and add dev comment Co-authored-by: あく <alleteam@gmail.com>
2022-01-29 12:20:41 +03:00
def getHeader(self):
if self.cursor != 0 and len(self.lines) == 0:
raise Exception("Can't read header data: cursor not at 0 or file is empty")
# Read Filetype
key, value = self.readKeyValue()
if key != "Filetype":
raise Exception("Invalid Header: missing `Filetype`")
filetype = value
# Read Version
key, value = self.readKeyValue()
if key != "Version":
raise Exception("Invalid Header: missing `Version`")
version = int(value)
return filetype, version
def setHeader(self, filetype: str, version: int):
if self.cursor != 0 and len(self.lines) != 0:
raise Exception("Can't set header data: file is not empty")
self.writeKey("Filetype", filetype)
self.writeKey("Version", version)
def load(self, filename: str):
[FL-2263] Flasher service & RAM exec (#1006) * WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar #2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com> Co-authored-by: あく <alleteam@gmail.com>
2022-04-13 23:50:25 +03:00
with open(filename, "r") as file:
self.lines = file.readlines()
[FL-2183] [FL-2209] Dolphin Deeds, Level up, assets generation, refactoring (#965) * Desktop: cleanup headers * Get loader pubsub via record * [FL-2183] Dolphin refactoring 2022.01 * Restruct animations assets structure * Rename assets * Cleanup headers * Update Recording animation * Add BadBattery animation * Provide loader's pubsub via record * Fix load/unload animations * Scripts: add flipper format support, initial dolphin packager rework. Assets: internal and external dolphin. * Sync internal meta.txt and manifest.txt * Reorder, rename dolphin assets * Split essential generated assets * Add ReadMe for dolphin assets * Separate essential blocking animations * Scripts: full dolphin validation before packaging * Assets, Scripts: dolphin external resources packer * Github: update codeowners * Scripts: proper slots handling in dolphin animation meta * Scripts: correct frames enumeration and fix compiled assets. * [FL-2209] Add Dolphin Deeds points and many more * Remove excess frame_rate * Change dolphin assets directory * Scripts: add internal resource support to dolphin compiler * Scripts: add internal assets generation, renaming * Scripts: correct assert, renaming * Code cleanup, documentation, fixes * Update Levelup animations * Rename essential -> blocking * Fix Unlocked hint * Scripts: rewrite Templite compiller, replace regexps with token parser, split block types into code and variable blocks. Update dolphin templates. * Documentation: add key combos description and use information * Scripts: cleanup templit, more debug info and add dev comment Co-authored-by: あく <alleteam@gmail.com>
2022-01-29 12:20:41 +03:00
def save(self, filename: str):
with open(filename, "w", newline="\n") as file:
[FL-2263] Flasher service & RAM exec (#1006) * WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar #2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <who.just.the.doctor@gmail.com> Co-authored-by: あく <alleteam@gmail.com>
2022-04-13 23:50:25 +03:00
file.write("\n".join(self.lines))