on xenial help sign in
Optimize wallpapers for load times and memory use

Load wallpapers with the same sourceSize, set to a square with a side length based on the largest dimension of the screen resolution. This required quite a bit of "pass it on," but the results are worth it: this saves 60MB of RAM when using the default wallpaper on the BQ E4.5. Wallpaper.qml is updated to be smart enough to warn a developer when they use it correctly. Read on for extended discussion of why every wallpaper needed the same sourceSize and why it's a square based on the largest dimension of the screen resolution. Images are almost always the most memory-intensive part of a QML application. To reduce their memory usage, you can set their sourceSize property. Setting sourceSize tells Qt to scale down or crop the image as it is loaded[1]. However, there's a problem with sourceSize... Whenever sourceSize changes, the entire image file must be reloaded from disk and stored in memory. QML has an image cache, but it only stores images based on their exact sourceSize. An image reload for a smaller sourceSize is faster than one for a larger sourceSize[2], but not nearly enough to make a reload fast enough to do during a quick transition. There is always a user-noticeable effect[3]. With that in mind, it's important to do a few things to save both load times and memory use: 1. Set sourceSize as small as possible 2. Change sourceSize as rarely as possible 3. Make sourceSize the same on every instance of the same image source The best possible sourceSize for memory use would be the exact resolution of the largest screen that Lomiri is currently running on. This would prevent multiple instances of the same image from loading (one per screen) and use a pretty small footprint. However, that creates a problem on mobile. Devices rotate. The Shell rotates with them. When the Shell rotates, the sourceSize would likely need to change. That can cause the transition on rotation to take noticeably longer to complete, which is not acceptable. Instead, we can compromise a little further on memory. If we set all instances to use a square sourceSize with the square's edge length based on the largest edge of the largest screen, we avoid reloading the image for a rotation event and still save considerable memory in most situations. We save the most memory on the lowest-spec devices, which generally have lower resolution screens to match their low RAM size. An image taken with the BQ E4.5's camera is 3264x2448px. Its screen resolution is 540x960. If we set the wallpaper's sourceSize to 960x960, Qt implicitly scales our requested size so the loaded image's smallest edge matches 960px but retains its aspect ratio[1], changing our camera image to 1280x960. That's 85% fewer pixels to store in RAM. For a concrete example: I ran Lomiri with --mode=shell and sized it to 540x960. Without this patch it uses 213MB of RAM. With this patch it uses 153MB. Of course the RAM savings become less impressive if the user's wallpaper is smaller than the beefy 4096x2304 of xenial's warty-final-ubuntu.png. Devices with a larger resolution in any direction will have worse results compared to the number of pixels on the screen, too. Sorry 2:1 screen heroes. But even with the unfortunate facts of geometry, the memory savings will be higher than zero on any device which has a wallpaper larger than its largest screen dimension. [1] https://doc.qt.io/qt-5.12/qml-qtquick-image.html#sourceSize-prop "If the image's actual size is larger than the sourceSize, the image is scaled down. If only one dimension of the size is set to greater than 0, the other dimension is set in proportion to preserve the source image's aspect ratio." I was unable to verify this behavior with the Qt Creator QML profiler. I setsourceSize to an aspect ratio that did not match the original. Whether the image's fillMode was set to Pad or PreserveAspectCrop, the Image Load event listed the image being loaded with a scaled resolution. However, the image with fillMode: Pad appeared on screen as a cropped square if sourceSize was a square, despite the profiler stating the image being loaded in its original aspect ratio. I'm not sure what's going on. [2] At least for JPG and PNG, I didn't test any other formats. JPG is slower to load than PNG for a given resolution and has larger improvements based on the sourceSize. [3] The UI thread pauses if asynchronous is false, the background flashes on screen if it's true.

commita81421ac1fe5135b9cff710c3cd819aa1804c6e6 authorDalton Durst <dalton@ubports.com> dateWed 2 Jun 2021, 19:29 (5 years ago) parentc3211f7
15 files changed+202−4
qml/Components/Lockscreen.qml+2 qml/Components/Wallpaper.qml+57−2 qml/Components/WallpaperResolver.qml+1 qml/Greeter/CoverPage.qml+1 qml/Greeter/Greeter.qml+7 qml/Greeter/NarrowView.qml+3 qml/Greeter/WideView.qml+1 qml/Launcher/Drawer.qml+2 qml/Launcher/Launcher.qml+1 qml/Shell.qml+18−1 qml/Stage/Stage.qml+3 tests/data/unity/backgrounds/20x30.png tests/qmltests/CMakeLists.txt+1 tests/qmltests/Components/tst_Wallpaper.qml+104 tests/qmltests/tst_Shell.qml+1−1
qml/Components/Lockscreen.qml+2
@@ -1,5 +1,6 @@
11/*
22 * Copyright (C) 2013-2017 Canonical, Ltd.
3+ * Copyright (C) 2021 UBports Foundation
34 *
45 * This program is free software; you can redistribute it and/or modify
56 * it under the terms of the GNU General Public License as published by
@@ -53,6 +54,7 @@ Showable {
5354 property int maxPinLength: -1
5455
5556 property url background: ""
57+ property alias backgroundSourceSize: backgroundImage.sourceSize
5658 // Use this to put a black overlay above the background
5759 // 0: normal background, 1: black background
5860 property real darkenBackground: 0
qml/Components/Wallpaper.qml+57−2
@@ -1,5 +1,6 @@
11/*
22 * Copyright (C) 2013-2016 Canonical, Ltd.
3+ * Copyright (C) 2021 UBports Foundation
34 *
45 * This program is free software; you can redistribute it and/or modify
56 * it under the terms of the GNU General Public License as published by
@@ -20,12 +21,66 @@ import Ubuntu.Components 1.3
2021
2122Item {
2223 id: root
23− property url source
24+ objectName: "Wallpaper"
25+ property url source: ""
26+
27+ // sourceSize should be set to the largest width or height that this
28+ // Wallpaper will likely be set to.
29+ // In most cases inside Lomiri, this should be set to the width or height
30+ // of the shell, whichever is larger.
31+ // However, care should be taken to avoid changing sourceSize once
32+ // it's set.
33+ property real sourceSize: -1.0
2434
2535 Image {
36+ id: image
37+ objectName: "wallpaperImage"
2638 anchors.fill: parent
2739 fillMode: Image.PreserveAspectCrop
2840
29− source: root.source
41+ // Setting this to true stops the UI from pausing when the shell's
42+ // dimensions change but causes the image to flash on screen.
43+ asynchronous: false
44+
45+ property real oldSourceSize: 0.0
46+ property real intermediarySourceSize: {
47+ // If minSourceSize is defined by root's parent, it will be 0 until
48+ // the binding resolves. Otherwise, it'll be -1.0. Abuse this
49+ // behavior to avoid changing sourceSize more than once during
50+ // startup
51+ if (root.sourceSize === 0.0) {
52+ developerSanityTimer.start();
53+ // -1.0 signals that the wallpaper shouldn't render yet.
54+ return -1.0
55+ }
56+ developerSanityTimer.stop();
57+ if (root.sourceSize === -1.0 && root.source !== "") {
58+ console.warn(`${root.objectName}'s sourceSize is unset. It will use a lot of memory.`);
59+ return 0.0
60+ }
61+
62+ return root.sourceSize
63+ }
64+
65+ onIntermediarySourceSizeChanged: {
66+ if (oldSourceSize !== 0.0 && oldSourceSize !== -1.0 && root.source !== "") {
67+ console.warn(`${root.objectName} source size is changing from ${oldSourceSize} to ${intermediarySourceSize}. This will cause an expensive image reload.`);
68+ }
69+ oldSourceSize = intermediarySourceSize;
70+ }
71+
72+ sourceSize: Qt.size(intermediarySourceSize, intermediarySourceSize)
73+ source: intermediarySourceSize !== -1.0 ? root.source : ""
74+ }
75+
76+ Timer {
77+ id: developerSanityTimer
78+ interval: 2000
79+ running: false
80+ onTriggered: {
81+ if (root.sourceSize === 0.0 && root.source !== "") {
82+ console.warn(`${root.objectName} has not received its sourceSize yet. It will not render.`);
83+ }
84+ }
3085 }
3186}
qml/Components/WallpaperResolver.qml+1
@@ -1,5 +1,6 @@
11/*
22 * Copyright (C) 2015 Canonical, Ltd.
3+ * Copyright (C) 2021 UBports Foundation
34 *
45 * This program is free software; you can redistribute it and/or modify
56 * it under the terms of the GNU General Public License as published by
qml/Greeter/CoverPage.qml+1
@@ -26,6 +26,7 @@ Showable {
2626 property real dragHandleLeftMargin
2727 property real launcherOffset
2828 property alias background: greeterBackground.source
29+ property alias backgroundSourceSize: greeterBackground.sourceSize
2930 property alias hasCustomBackground: backgroundShade.visible
3031 property real panelHeight
3132 property var infographicModel
qml/Greeter/Greeter.qml+7
@@ -35,6 +35,7 @@ Showable {
3535
3636 property url background
3737 property bool hasCustomBackground
38+ property real backgroundSourceSize
3839
3940 // How far to offset the top greeter layer during a launcher left-drag
4041 property real launcherOffset
@@ -412,6 +413,12 @@ Showable {
412413 value: root.background
413414 }
414415
416+ Binding {
417+ target: loader.item
418+ property: "backgroundSourceSize"
419+ value: root.backgroundSourceSize
420+ }
421+
415422 Binding {
416423 target: loader.item
417424 property: "hasCustomBackground"
qml/Greeter/NarrowView.qml+3
@@ -30,6 +30,7 @@ FocusScope {
3030 property alias delayMinutes: delayedLockscreen.delayMinutes
3131 property real panelHeight
3232 property url background
33+ property real backgroundSourceSize
3334 property bool hasCustomBackground
3435 property bool locked
3536 property alias alphanumeric: loginList.alphanumeric
@@ -113,6 +114,7 @@ FocusScope {
113114 fill: parent
114115 }
115116 source: root.background
117+ sourceSize: root.backgroundSourceSize
116118 }
117119
118120 // Darken background to match CoverPage
@@ -175,6 +177,7 @@ FocusScope {
175177 draggable: !root.waiting
176178 onTease: root.tease()
177179 onClicked: hide()
180+ backgroundSourceSize: root.backgroundSourceSize
178181
179182 onShowProgressChanged: {
180183 if (showProgress === 0) {
qml/Greeter/WideView.qml+1
@@ -26,6 +26,7 @@ FocusScope {
2626 focus: true
2727
2828 property alias background: coverPage.background
29+ property alias backgroundSourceSize: coverPage.backgroundSourceSize
2930 property alias panelHeight: coverPage.panelHeight
3031 property alias hasCustomBackground: coverPage.hasCustomBackground
3132 property alias dragHandleLeftMargin: coverPage.dragHandleLeftMargin
qml/Launcher/Drawer.qml+2
@@ -33,6 +33,7 @@ FocusScope {
3333 readonly property Item searchTextField: searchField
3434 readonly property real delegateWidth: units.gu(10)
3535 property url background
36+ property alias backgroundSourceSize: background.sourceSize
3637 visible: x > -width
3738 property var fullyOpen: x === 0
3839 property var fullyClosed: x === -width
@@ -159,6 +160,7 @@ FocusScope {
159160
160161 Wallpaper {
161162 id: background
163+ objectName: "drawerBackground"
162164 anchors.fill: parent
163165 source: root.background
164166 }
qml/Launcher/Launcher.qml+1
@@ -35,6 +35,7 @@ FocusScope {
3535 property bool drawerEnabled: true
3636 property alias privateMode: panel.privateMode
3737 property url background
38+ property alias backgroundSourceSize: drawer.backgroundSourceSize
3839
3940 property int panelWidth: units.gu(10)
4041 property int dragAreaWidth: units.gu(1)
qml/Shell.qml+18−1
@@ -15,7 +15,7 @@
1515 * along with this program. If not, see <http://www.gnu.org/licenses/>.
1616 */
1717
18−import QtQuick 2.4
18+import QtQuick 2.8
1919import QtQuick.Window 2.2
2020import AccountsService 0.1
2121import Unity.Application 0.1
@@ -71,6 +71,20 @@ StyledItem {
7171 property bool hasTouchscreen: false
7272 property bool supportsMultiColorLed: true
7373
74+ // The largest dimension, in pixels, of all of the screens this Shell is
75+ // operating on.
76+ // If a script sets the shell to 240x320 when it was 320x240, we could
77+ // end up in a situation where our dimensions are 240x240 for a short time.
78+ // Notifying the Wallpaper of both events would make it reload the image
79+ // twice. So, we use a Binding { delayed: true }.
80+ property real largestScreenDimension
81+ Binding {
82+ target: shell
83+ delayed: true
84+ property: "largestScreenDimension"
85+ value: Math.max(nativeWidth, nativeHeight)
86+ }
87+
7488 // Used by tests
7589 property alias lightIndicators: indicatorsModel.light
7690
@@ -308,6 +322,7 @@ StyledItem {
308322
309323 dragAreaWidth: shell.edgeSize
310324 background: wallpaperResolver.background
325+ backgroundSourceSize: shell.largestScreenDimension
311326
312327 applicationManager: ApplicationManager
313328 topLevelSurfaceList: topLevelSurfaceList
@@ -423,6 +438,7 @@ StyledItem {
423438 tabletMode: shell.usageScenario != "phone"
424439 forcedUnlock: wizard.active || shell.mode === "full-shell"
425440 background: wallpaperResolver.background
441+ backgroundSourceSize: shell.largestScreenDimension
426442 hasCustomBackground: wallpaperResolver.hasCustomBackground
427443 allowFingerprint: !dialogs.hasActiveDialog &&
428444 !notifications.topmostIsFullscreen &&
@@ -588,6 +604,7 @@ StyledItem {
588604 drawerEnabled: !greeter.active && tutorial.launcherLongSwipeEnabled
589605 privateMode: greeter.active
590606 background: wallpaperResolver.background
607+ backgroundSourceSize: shell.largestScreenDimension
591608
592609 // It can be assumed that the Launcher and Panel would overlap if
593610 // the Panel is open and taking up the full width of the shell
qml/Stage/Stage.qml+3
@@ -1,5 +1,6 @@
11/*
22 * Copyright (C) 2014-2017 Canonical, Ltd.
3+ * Copyright (C) 2021 UBports Foundation
34 *
45 * This program is free software; you can redistribute it and/or modify
56 * it under the terms of the GNU General Public License as published by
@@ -36,6 +37,7 @@ FocusScope {
3637 property QtObject topLevelSurfaceList
3738 property bool altTabPressed
3839 property url background
40+ property alias backgroundSourceSize: wallpaper.sourceSize
3941 property int dragAreaWidth
4042 property real nativeHeight
4143 property real nativeWidth
@@ -625,6 +627,7 @@ FocusScope {
625627
626628 Wallpaper {
627629 id: wallpaper
630+ objectName: "stageBackground"
628631 anchors.fill: parent
629632 source: root.background
630633 // Make sure it's the lowest item. Due to the left edge drag we sometimes need
tests/data/unity/backgrounds/20x30.pngnew file

Binary file not shown.

tests/qmltests/CMakeLists.txt+1
@@ -23,6 +23,7 @@ add_unity8_qmltest(Components Rating)
2323add_unity8_qmltest(Components SharingPicker)
2424add_unity8_qmltest(Components Showable)
2525add_unity8_qmltest(Components VirtualTouchPad)
26+add_unity8_qmltest(Components Wallpaper)
2627add_unity8_qmltest(Components WallpaperResolver)
2728add_unity8_qmltest(Components ZoomableImage)
2829add_unity8_qmltest(Greeter Clock)
tests/qmltests/Components/tst_Wallpaper.qmlnew file+104
@@ -0,0 +1,104 @@
1+/*
2+ * Copyright (C) 2021 UBports Foundation
3+ *
4+ * This program is free software; you can redistribute it and/or modify
5+ * it under the terms of the GNU General Public License as published by
6+ * the Free Software Foundation; version 3.
7+ *
8+ * This program is distributed in the hope that it will be useful,
9+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
10+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
11+ * GNU General Public License for more details.
12+ *
13+ * You should have received a copy of the GNU General Public License
14+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
15+ */
16+
17+import QtQuick 2.4
18+import QtTest 1.0
19+import Ubuntu.Components 1.3
20+import Unity.Test 0.1
21+
22+import "../../../qml/Components"
23+
24+
25+Item {
26+ id: root
27+ width: 150
28+ height: 100
29+
30+ property url defaultBackground: "../../data/unity/backgrounds/20x30.png"
31+ property real sourceSize: 0
32+
33+ Component {
34+ id: wallpaperComponent
35+ Wallpaper {
36+ anchors.fill: parent
37+ source: root.defaultBackground
38+ sourceSize: root.sourceSize
39+ }
40+ }
41+
42+ Component {
43+ id: unsetSourceSizeWallpaperComponent
44+ Wallpaper {
45+ anchors.fill: parent
46+ source: root.defaultBackground
47+ }
48+ }
49+
50+ UnityTestCase {
51+ id: testCase
52+ name: "Wallpaper"
53+ when: windowShown
54+
55+ function init() {
56+ root.sourceSize = 0;
57+ }
58+
59+ // Ensures that the Wallpaper's sourceSize causes the image to be
60+ // scaled while matching its original aspect ratio.
61+ function test_wallpaperGetsSourceSize() {
62+ root.sourceSize = 10;
63+ var wallpaper = createTemporaryObject(wallpaperComponent, root);
64+ var image = findChild(wallpaper, "wallpaperImage");
65+ verify(image);
66+ tryCompare(image, "status", Image.Ready);
67+ tryCompare(image, "implicitWidth", 10.0);
68+ tryCompare(image, "implicitHeight", 15.0);
69+ }
70+
71+ function test_warningOnSizeChange() {
72+ root.sourceSize = 1;
73+ var wallpaper = createTemporaryObject(wallpaperComponent, root);
74+ ignoreWarning(new RegExp(".*source size is changing.*"));
75+ root.sourceSize = 10;
76+ }
77+
78+ // Ensures the Wallpaper's sourceSize is unbounded and the image loads
79+ // when sourceSize is unset
80+ function test_sizeUnset() {
81+ ignoreWarning(new RegExp(".*sourceSize is unset.*"));
82+ var wallpaper = createTemporaryObject(unsetSourceSizeWallpaperComponent, root);
83+ var image = findChild(wallpaper, "wallpaperImage");
84+ // Reading the property is enough to make sure the warning happened
85+ tryCompare(image, "status", Image.Ready);
86+ tryCompare(image, "implicitWidth", 20.0);
87+ tryCompare(image, "implicitHeight", 30.0);
88+ }
89+
90+ function test_warningOnSizeBindingFailed() {
91+ // When a binding fails, it will be undefined or zero. Since
92+ // a 'real' property is read as zero when it is undefined anyway,
93+ // use that.
94+ root.sourceSize = 0;
95+ ignoreWarning(new RegExp(".*will not render.*"));
96+ var wallpaper = createTemporaryObject(wallpaperComponent, root);
97+ // A SignalSpy waiting for the signal that causes the warning would
98+ // be more correct, but hooking up the SignalSpy is sometimes slower
99+ // than the event triggering the warning and we end up waiting
100+ // forever.
101+ wait(3000);
102+ }
103+ }
104+}
tests/qmltests/tst_Shell.qml+1−1
@@ -1,6 +1,6 @@
11/*
22 * Copyright (C) 2013-2016 Canonical, Ltd.
3− * Copyright (C) 2019 UBports Foundation
3+ * Copyright (C) 2019-2021 UBports Foundation
44 *
55 * This program is free software; you can redistribute it and/or modify
66 * it under the terms of the GNU General Public License as published by