Merge pull request #149 from Nuxij/unified-dockerfile

Unify the Dockerfiles by fleshing out build.sh
2024-11-27 10:43:46 +03:00 · 2020-12-23 15:20:20 +01:00 · 2020-12-23 15:20:20 +01:00 · b4fb8897ac
commit b4fb8897ac
parent 276c8aa308 5cdd3f6e6f
8 changed files with 190 additions and 136 deletions
--- a/.gitignore
+++ b/.gitignore
@ -1,11 +1,13 @@
 .idea/
 # CMake
-cmake-build-*/
-CMakeFiles/
+cmake-build-*
+cmake-*
+CMakeFiles
 **/CMakeCache.txt
 cmake_install.cmake
 Makefile
-build/
+build
+tools

 # Resulting binary files
 *.a
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@ -65,7 +65,7 @@ endif()

 set(VERSION_EDIT_WARNING "// Do not edit this file, it is automatically generated by CMAKE!")
 configure_file(${CMAKE_CURRENT_SOURCE_DIR}/src/Version.h.in ${CMAKE_CURRENT_SOURCE_DIR}/src/Version.h)
-configure_file(${CMAKE_CURRENT_SOURCE_DIR}/docker/post_build.sh.in ${CMAKE_CURRENT_SOURCE_DIR}/docker/post_build.sh)
+configure_file(${CMAKE_CURRENT_SOURCE_DIR}/docker/post_build.sh.in ${CMAKE_CURRENT_BINARY_DIR}/post_build.sh)


 add_subdirectory(src)
--- a/doc/buildWithDocker.md
+++ b/doc/buildWithDocker.md
@ -1,13 +1,52 @@
 # Build the project using Docker
-There are [Docker images (Dockerfile)](../docker) containing the build environment for AMD64 (x86_64) and ARM64 architectures. These images make the build of the firmware and the generation of the DFU file for OTA quite easy, as well as preventing clashes with any other toolchains or development environments you may have installed.
+
+A [Docker image (Dockerfile)](../docker) containing all the build environment is available for X86_64 and AMD64 architectures. These images make the build of the firmware and the generation of the DFU file for OTA quite easy, as well as preventing clashes with any other toolchains or development environments you may have installed.
+
+Based on Ubuntu 18.04 with the following build dependencies:
+
+* ARM GCC Toolchain
+* nRF SDK
+* MCUBoot
+* adafruit-nrfutil
+
+## Run a container to build the project
+
+The `infinitime-build` image contains all the dependencies you need. The default `CMD` will compile sources found in `/sources`, so you need only mount your code. 
+
+This example  will build the firmware, generate the MCUBoot image and generate the DFU file. Outputs will be written to **<project_root>/build/output**:
+
+```bash
+cd <project_root> # e.g. cd ./work/Pinetime
+docker run --rm -it -v $(pwd):/sources infinitime-build
+```
+
+If you only want to build a single CMake target, you can pass it in as the first parameter to the build script. This means calling the script explicitly as it will override the `CMD`. Here's an example For `pinetime-app`:
+
+```bash
+docker run --rm -it -v $(pwd):/sources infinitime-build /opt/build.sh pinetime-app
+```
+
+The image is built using 1000:1000 for the user id and group id. If this is different to your user or group ids (run `id -u` and `id -g` to find out what your id values are if you are unsure), you will need to override them via the `--user` parameter in order to prevent permission errors with the output files (and the cmake build cache).
+
+Running with this image is the same as above, you just specify the ids to `docker run`
+
+```bash
+docker run --rm -it -v $(pwd):/sources --user $(id -u):$(id -g) pfeerick/infinitime-build
+```
+
+Or you can specify your user id and group id (by number, not by name) directly:
+
+```bash
+docker run --rm -it -v $(pwd):/sources --user 1234:1234 infinitime-build
+```

 ## Using the image from Docker Hub

-The image is avaiable via Docker Hub for both the amd64 and arm64v8 architectures at [pfeerick/infinitime-build](https://hub.docker.com/repository/docker/pfeerick/infinitime-build). 
+The image is avaiable via Docker Hub for both the amd64 and arm64v8 architectures at [pfeerick/infinitime-build](https://hub.docker.com/repository/docker/pfeerick/infinitime-build).

 It can be pulled (downloaded) using the following command:

-```
+```bash
 docker pull pfeerick/infinitime-build
 ```

@ -17,53 +56,18 @@ The default `latest` tag *should* automatically identify the correct image archi

 * For ARM64v8 (ARM64/aarch64) systems: `docker pull pfeerick/infinitime-build:arm64v8`

-The Docker Hub images are built using 1000:1000 for the user id and group id. If this is different to your user or group ids (run `id -u` and `id -g` to find out what your id values are if you are unsure), you will need to override them via the `--user` parameter in order to prevent permission errors during and after compilation.
+## Build the image

-The below example will run the container, setting the user and group ids automatically:
+You can build the image yourself if you like!

+The following commands must be run from the root of the project. This operation will take some time but, when done, a new image named *infinitime-build* is available.
+
+```bash
+docker image build -t infinitime-build ./docker
 ```
-docker run --rm -v <project_root>:/sources --user $(id -u):$(id -g) pfeerick/infinitime-build
-```
-
-Or you can specify your user id and group id (by number, not by name) directly:
-
-```
-docker run --rm -v <project_root>:/sources --user uid_num:gid_num pfeerick/infinitime-build
-```
-
-## Build the image yourself
-Building the docker images yourself is quite easy. The following commands must be run from the root of the project.

 The `PUID` and `PGID` build arguments are used to set the user and group ids used in the container, meaning you will not need to specify it later unless they change for some reason. Specifying them is not mandatory, as this can be over-ridden at build time via the `--user` flag, but doing so will make the command you need to run later a bit shorter. In the below examples, they are set to your current user id and group id automatically. You can specify them manually, but they must be specified by number, not by name.

-If you are running on a AMD64 (x86_64) computer: 
+```bash
+docker image build -t infinitime-build --build-arg PUID=$(id -u) --build-arg PGID=$(id -g) ./docker
 ```
-docker image build -t infinitime-build --build-arg PUID=$(id -u) --build-arg PGID=$(id -g) docker/amd64/
-```
-
-If you are running on an ARM64 computer (tested on Raspberry Pi 4 and Pine64 Pinebook Pro):
-```
-docker image build -t infinitime-build --build-arg PUID=$(id -u) --build-arg PGID=$(id -g) docker/arm64v8/
-```
-
-This operation will take some time, as it builds a Docker image based on Ubuntu, installs some required packages, downloads the ARM toolchain, the NRF SDK, MCUBoot and adafruit-nrfutil.
-
-When this is done, a new image named *infinitime-build* is available.
-
-## Run a container to build the project:
-
-The command to run the container is essentially the same, regardless of whether you built it yourself from the dockerfiles, or are using the Docker Hub images (use `pfeerick/infinitime-build` instead of `infinitime-build` for the later):
-
-```
-docker run --rm -v <project_root>:/sources infinitime-build
-```
-
-This will start a container (removing it when finished), build the firmware and generate the MCUBoot image and DFU file. The output of the build is stored in `<project_root>/built/output`.
-
-Replace *<project_root>* by the path of the root of the project on your computer. For example:
-
-```
-docker run --rm -v /home/jf/git/PineTime:/sources infinitime-build
-```
-
-If you encounter permission errors (due to being logged in as a different user, changed user id, running the docker hub image, etc.), see the `--user` parameter mentioned above in the Docker Hub image section to see if this resolves the issue for you.
--- a/docker/Dockerfile
+++ b/docker/Dockerfile
@ -0,0 +1,42 @@
+FROM ubuntu:18.04
+
+ARG DEBIAN_FRONTEND=noninteractive
+RUN apt-get update -qq \
+    && apt-get install -y \
+# x86_64 / generic packages
+      bash \
+      build-essential \
+      cmake \
+      git \
+      make \
+      python3 \
+      python3-pip \
+      tar \
+      unzip \
+      wget \ 
+# aarch64 packages
+      libffi-dev \
+      libssl-dev \
+      python3-dev \ 
+    && rm -rf /var/cache/apt/* /var/lib/apt/lists/*;
+
+RUN pip3 install adafruit-nrfutil
+
+# build.sh knows how to compile
+COPY build.sh /opt/
+
+# Lets get each in a separate docker layer for better downloads
+# GCC
+RUN bash -c "source /opt/build.sh; GetGcc;"
+# NrfSdk
+RUN bash -c "source /opt/build.sh; GetNrfSdk;"
+# McuBoot
+RUN bash -c "source /opt/build.sh; GetMcuBoot;"
+
+ARG PUID=1000
+ARG PGID=1000
+RUN groupadd --system --gid $PGID infinitime && useradd --system --uid $PUID --gid $PGID infinitime
+
+USER infinitime:infinitime
+ENV SOURCES_DIR /sources
+CMD ["/opt/build.sh"]
--- a/docker/amd64/Dockerfile
+++ b/docker/amd64/Dockerfile
@ -1,34 +0,0 @@
-FROM amd64/ubuntu:18.04
-
-ARG DEBIAN_FRONTEND=noninteractive
-RUN apt-get update -qq \
-    && apt-get install -y \ 
-        build-essential \
-        cmake \
-        git \
-        make \
-        python3 \
-        python3-pip \
-        tar \
-        unzip \
-        wget \
-    && rm -rf /var/cache/apt/* /var/lib/apt/lists/*
-
-RUN wget -q https://developer.arm.com/-/media/Files/downloads/gnu-rm/9-2020q2/gcc-arm-none-eabi-9-2020-q2-update-x86_64-linux.tar.bz \
-  && tar -xjf gcc-arm-none-eabi-9-2020-q2-update-x86_64-linux.tar.bz -C /opt \
-  && rm gcc-arm-none-eabi-9-2020-q2-update-x86_64-linux.tar.bz
-
-RUN wget -q https://developer.nordicsemi.com/nRF5_SDK/nRF5_SDK_v15.x.x/nRF5_SDK_15.3.0_59ac345.zip  \
-  && unzip -q nRF5_SDK_15.3.0_59ac345.zip -d /opt/ && rm nRF5_SDK_15.3.0_59ac345.zip
-
-RUN git clone https://github.com/JuulLabs-OSS/mcuboot.git /opt/mcuboot \
-  && pip3 install -r /opt/mcuboot/scripts/requirements.txt
-
-RUN pip3 install adafruit-nrfutil
-
-ARG PUID=1000
-ARG PGID=1000
-RUN groupadd --system --gid $PGID infinitime && useradd --system --uid $PUID --gid $PGID infinitime
-
-USER infinitime:infinitime
-CMD ["/sources/docker/build.sh"]
--- a/docker/arm64v8/Dockerfile
+++ b/docker/arm64v8/Dockerfile
@ -1,37 +0,0 @@
-FROM arm64v8/ubuntu:18.04
-
-ENV DEBIAN_FRONTEND=noninteractive
-RUN apt-get update -qq \
-    && apt-get install -y \ 
-        build-essential \
-        cmake \
-        git \
-        libffi-dev \
-        libssl-dev \
-        make \
-        python3 \
-        python3-dev \
-        python3-pip \
-        tar \
-        unzip \
-        wget \
-    && rm -rf /var/cache/apt/* /var/lib/apt/lists/*
-
-RUN wget -q https://developer.arm.com/-/media/Files/downloads/gnu-rm/9-2020q2/gcc-arm-none-eabi-9-2020-q2-update-aarch64-linux.tar.bz2 \
-  && tar -xjf gcc-arm-none-eabi-9-2020-q2-update-aarch64-linux.tar.bz2 -C /opt \
-  && rm gcc-arm-none-eabi-9-2020-q2-update-aarch64-linux.tar.bz2
-
-RUN wget -q https://developer.nordicsemi.com/nRF5_SDK/nRF5_SDK_v15.x.x/nRF5_SDK_15.3.0_59ac345.zip \
-  && unzip -q nRF5_SDK_15.3.0_59ac345.zip -d /opt/ \
-  && rm nRF5_SDK_15.3.0_59ac345.zip
-
-RUN git clone https://github.com/JuulLabs-OSS/mcuboot.git /opt/mcuboot && pip3 install -r /opt/mcuboot/scripts/requirements.txt
-
-RUN pip3 install adafruit-nrfutil
-
-ARG PUID=1000
-ARG PGID=1000
-RUN groupadd --system --gid $PGID infinitime && useradd --system --uid $PUID --gid $PGID infinitime
-
-USER infinitime:infinitime
-CMD ["/sources/docker/build.sh"]
--- a/docker/build.sh
+++ b/docker/build.sh
@ -1,12 +1,76 @@
-#!/bin/sh
+#!/bin/bash
+(return 0 2>/dev/null) && SOURCED="true" || SOURCED="false"
 export LC_ALL=C.UTF-8
 export LANG=C.UTF-8
 set -x
+set -e

-mkdir /sources/build
-cd /sources/build
+# Default locations if the var isn't already set
+export TOOLS_DIR="${TOOLS_DIR:=/opt}"
+export SOURCES_DIR="${SOURCES_DIR:=/sources}"
+export BUILD_DIR="${BUILD_DIR:=$SOURCES_DIR/build}"
+export OUTPUT_DIR="${OUTPUT_DIR:=$BUILD_DIR/output}"

-cmake -DARM_NONE_EABI_TOOLCHAIN_PATH=/opt/gcc-arm-none-eabi-9-2020-q2-update -DNRF5_SDK_PATH=/opt/nRF5_SDK_15.3.0_59ac345 -DUSE_OPENOCD=1 ../
-make -j$(nproc)
+export BUILD_TYPE=${BUILD_TYPE:=Release}
+export GCC_ARM_VER=${GCC_ARM_VER:="gcc-arm-none-eabi-9-2020-q2-update"}
+export NRF_SDK_VER=${NRF_SDK_VER:="nRF5_SDK_15.3.0_59ac345"}

-sh /sources/docker/post_build.sh
+MACHINE="$(uname -m)"
+[[ "$MACHINE" == "arm64" ]] && MACHINE="aarch64"
+
+main() {
+  local target="$1"
+
+  mkdir -p "$TOOLS_DIR"
+  
+  [[ ! -d "$TOOLS_DIR/$GCC_ARM_VER" ]] && GetGcc
+  [[ ! -d "$TOOLS_DIR/$NRF_SDK_VER" ]] && GetNrfSdk
+  [[ ! -d "$TOOLS_DIR/mcuboot" ]] && GetMcuBoot
+
+  mkdir -p "$BUILD_DIR"
+
+  CmakeGenerate
+  CmakeBuild $target
+
+  if [[ "$DISABLE_POSTBUILD" != "true" ]]; then
+    source "$BUILD_DIR/post_build.sh"
+  fi
+}
+
+GetGcc() {
+  GCC_SRC="$GCC_ARM_VER-$MACHINE-linux.tar.bz"
+  wget -q https://developer.arm.com/-/media/Files/downloads/gnu-rm/9-2020q2/$GCC_SRC -O - | tar -xj -C $TOOLS_DIR/
+}
+
+GetMcuBoot() {
+  git clone https://github.com/JuulLabs-OSS/mcuboot.git "$TOOLS_DIR/mcuboot"
+  pip3 install -r "$TOOLS_DIR/mcuboot/scripts/requirements.txt"
+}
+
+GetNrfSdk() {
+  wget -q "https://developer.nordicsemi.com/nRF5_SDK/nRF5_SDK_v15.x.x/$NRF_SDK_VER.zip" -O /tmp/$NRF_SDK_VER
+  unzip -q /tmp/$NRF_SDK_VER -d "$TOOLS_DIR/"
+  rm /tmp/$NRF_SDK_VER
+}
+
+CmakeGenerate() {
+  # We can swap the CD and trailing SOURCES_DIR for -B and -S respectively
+  # once we go to newer CMake (Ubuntu 18.10 gives us CMake 3.10)
+  cd "$BUILD_DIR"
+
+  cmake -G "Unix Makefiles" \
+    -DCMAKE_BUILD_TYPE=$BUILD_TYPE \
+    -DUSE_OPENOCD=1 \
+    -DARM_NONE_EABI_TOOLCHAIN_PATH="$TOOLS_DIR/$GCC_ARM_VER" \
+    -DNRF5_SDK_PATH="$TOOLS_DIR/$NRF_SDK_VER" \
+    "$SOURCES_DIR"
+  cmake -L -N .
+}
+
+CmakeBuild() {
+  local target="$1"
+  [[ -n "$target" ]] && target="--target $target"
+  cmake --build "$BUILD_DIR" --config $BUILD_TYPE $target -- -j$(nproc)
+}
+
+[[ $SOURCED == "false" ]] && main "$@" || echo "Sourced!"
--- a/docker/post_build.sh.in
+++ b/docker/post_build.sh.in
@ -1,16 +1,29 @@
 #!/bin/sh
 export LC_ALL=C.UTF-8
 export LANG=C.UTF-8
-set -x
+set -e
+set +x

-mkdir -p /sources/build/output
-/opt/mcuboot/scripts/imgtool.py create --align 4 --version 1.0.0 --header-size 32 --slot-size 475136 --pad-header /sources/build/src/pinetime-mcuboot-app-@PROJECT_VERSION_MAJOR@.@PROJECT_VERSION_MINOR@.@PROJECT_VERSION_PATCH@.bin /sources/build/output/image-@PROJECT_VERSION_MAJOR@.@PROJECT_VERSION_MINOR@.@PROJECT_VERSION_PATCH@.bin
-adafruit-nrfutil dfu genpkg --dev-type 0x0052 --application /sources/build/output/image-@PROJECT_VERSION_MAJOR@.@PROJECT_VERSION_MINOR@.@PROJECT_VERSION_PATCH@.bin /sources/build/output/dfu-@PROJECT_VERSION_MAJOR@.@PROJECT_VERSION_MINOR@.@PROJECT_VERSION_PATCH@.zip
+export PROJECT_VERSION="@PROJECT_VERSION_MAJOR@.@PROJECT_VERSION_MINOR@.@PROJECT_VERSION_PATCH@"

-cp /sources/build/src/*.bin /sources/build/output/
-cp /sources/build/src/*.hex /sources/build/output/
-cp /sources/build/src/*.out /sources/build/output/
-cp /sources/build/src/*.map /sources/build/output/
-cp /sources/bootloader/bootloader-5.0.4.bin /sources/build/output/bootloader.bin
+mkdir -p "$OUTPUT_DIR"

+cp "$SOURCES_DIR"/bootloader/bootloader-5.0.4.bin $OUTPUT_DIR/bootloader.bin

+"$TOOLS_DIR"/mcuboot/scripts/imgtool.py create --version 1.0.0 \
+    --align 4 --header-size 32 --slot-size 475136 --pad-header \
+    "$BUILD_DIR/src/pinetime-mcuboot-app-$PROJECT_VERSION.bin" \
+    "$OUTPUT_DIR/image-$PROJECT_VERSION.bin"
+
+adafruit-nrfutil dfu genpkg --dev-type 0x0052 \
+    --application "$OUTPUT_DIR/image-$PROJECT_VERSION.bin" \
+    "$OUTPUT_DIR/dfu-$PROJECT_VERSION.zip"
+
+mkdir -p "$OUTPUT_DIR/src"
+cd "$BUILD_DIR"
+cp src/*.bin "$OUTPUT_DIR/src"
+cp src/*.hex "$OUTPUT_DIR/src"
+cp src/*.out "$OUTPUT_DIR/src"
+cp src/*.map "$OUTPUT_DIR/src"
+
+ls -RUv1 "$OUTPUT_DIR" | sed 's;^\([^/]\); \1;g'