Initial Linux nightly bundles upload (#8913)

Changes Zed CI to build and upload Linux nightly bundles.

* `todo!(linux)` are replaced with `TODO linux` to make `todo!`-based
workflows more convenient
* renames `run-build-dmg` label into `run-bundling`, also renames a few
GH Actions entries to be more generic
* make another upload path for Linux, which keeps a separate file with SHA to version the nightly artifact.
* adds a `*.deb` package building with a couple of caveats, marked with
new `TODO linux` entries:

1. `cargo-bundle` is not very flexible, so it generates artifacts with
the structure and names that we're unable to alter before/during the
generation.
For that, a set of extra steps is made by repacking the *.deb package —
this is not very portable between different Linux distros, so later one
needs to find a way to combine multiple package types in this script.

2. `cargo-bundle` is not able to properly generate the *.msi bundle
despite declaring it in the features:
https://github.com/burtonageo/cargo-bundle/issues/116
Windows needs to invent its own way of bundling or fix the tool.

3. Both `cli` and `zed` binaries are added into the archive under
`/usr/local/bin/` path with their `-$channel` suffix
(-nightly/-preview/-dev/-stable) and a `/usr/local/bin/zed ->
/usr/local/bin/cli-nightly` symlink is made to make CLI work as Zed
launcher:
```
~/work/zed kb/linux-nightly:origin/kb/linux-nightly*​ ❯ dpkg -c target/zed_amd64.deb 
drwxr-xr-x allaptop/allaptop 0 2024-03-06 00:53 ./
drwxr-xr-x allaptop/allaptop 0 2024-03-06 00:53 ./usr/
drwxr-xr-x allaptop/allaptop 0 2024-03-06 00:53 ./usr/local/
drwxr-xr-x allaptop/allaptop 0 2024-03-06 00:53 ./usr/local/bin/
-rwxr-xr-x allaptop/allaptop 8746832 2024-03-06 00:53 ./usr/local/bin/cli-nightly
-rwxr-xr-x allaptop/allaptop 689078560 2024-03-06 00:53 ./usr/local/bin/zed-nightly
drwxr-xr-x allaptop/allaptop         0 2024-03-06 00:53 ./usr/share/
drwxr-xr-x allaptop/allaptop         0 2024-03-06 00:53 ./usr/share/applications/
-rw-r--r-- allaptop/allaptop       153 2024-03-06 00:53 ./usr/share/applications/zed.desktop
drwxr-xr-x allaptop/allaptop         0 2024-03-06 00:53 ./usr/share/icons/
drwxr-xr-x allaptop/allaptop         0 2024-03-06 00:53 ./usr/share/icons/hicolor/
drwxr-xr-x allaptop/allaptop         0 2024-03-06 00:53 ./usr/share/icons/hicolor/1024x1024@2x/
drwxr-xr-x allaptop/allaptop         0 2024-03-06 00:53 ./usr/share/icons/hicolor/1024x1024@2x/apps/
-rw-r--r-- allaptop/allaptop    716288 2024-03-06 00:53 ./usr/share/icons/hicolor/1024x1024@2x/apps/zed.png
drwxr-xr-x allaptop/allaptop         0 2024-03-06 00:53 ./usr/share/icons/hicolor/512x512/
drwxr-xr-x allaptop/allaptop         0 2024-03-06 00:53 ./usr/share/icons/hicolor/512x512/apps/
-rw-r--r-- allaptop/allaptop    239870 2024-03-06 00:53 ./usr/share/icons/hicolor/512x512/apps/zed.png
lrwxrwxrwx allaptop/allaptop         0 2024-03-06 00:53 ./usr/local/bin/zed -> /usr/local/bin/cli-nightly
```

But the CLI does not work under Linux yet and there's no way to install
that CLI from Zed now; Zed binary itself is not able to open
`file/location:12:34`-like things and set up the env properly, but is
able to start or open a directory.

So, this structure can be considered temporary and changed, if needed.

4. Zed Nightly on Linux does not know how to update itself, so all
nightly publishing is not picked up automatically.

5. Rust cache from `main` builds does not get shared between CI jobs,
due to being run in a different CI job that forms a different CI key, so
```
      - name: Cache dependencies
        uses: swatinem/rust-cache@v2
        with:
          save-if: ${{ false }}
```
would not work.
This makes Linux bundling jobs long.

Release Notes:

- N/A

.github/workflows/ci.yml

@@ -152,12 +152,12 @@ jobs:
       - name: Build Zed
         run: cargo build -p zed
 
-  bundle:
-    name: Bundle macOS app
+  bundle-mac:
+    name: Create a macOS bundle
     runs-on:
       - self-hosted
       - bundle
-    if: ${{ startsWith(github.ref, 'refs/tags/v') || contains(github.event.pull_request.labels.*.name, 'run-build-dmg') }}
+    if: ${{ startsWith(github.ref, 'refs/tags/v') || contains(github.event.pull_request.labels.*.name, 'run-bundling') }}
     needs: [macos_tests]
     env:
       MACOS_CERTIFICATE: ${{ secrets.MACOS_CERTIFICATE }}
@@ -212,12 +212,12 @@ jobs:
       - name: Generate license file
         run: script/generate-licenses
 
-      - name: Create app bundle
-        run: script/bundle
+      - name: Create macOS app bundle
+        run: script/bundle-mac
 
       - name: Upload app bundle to workflow run if main branch or specific label
-        uses: actions/upload-artifact@v3
-        if: ${{ github.ref == 'refs/heads/main' }} || contains(github.event.pull_request.labels.*.name, 'run-build-dmg') }}
+        uses: actions/upload-artifact@v4
+        if: ${{ github.ref == 'refs/heads/main' }} || contains(github.event.pull_request.labels.*.name, 'run-bundling') }}
         with:
           name: Zed_${{ github.event.pull_request.head.sha || github.sha }}.dmg
           path: target/release/Zed.dmg
@@ -232,3 +232,81 @@ jobs:
           body: ""
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+  bundle-deb:
+    name: Create a *.deb Linux bundle
+    runs-on: ubuntu-22.04 # keep the version fixed to avoid libc and dynamic linked library issues
+    if: ${{ startsWith(github.ref, 'refs/tags/v') || contains(github.event.pull_request.labels.*.name, 'run-bundling') }}
+    needs: [linux_tests]
+    env:
+      ZED_CLIENT_CHECKSUM_SEED: ${{ secrets.ZED_CLIENT_CHECKSUM_SEED }}
+      DIGITALOCEAN_SPACES_ACCESS_KEY: ${{ secrets.DIGITALOCEAN_SPACES_ACCESS_KEY }}
+      DIGITALOCEAN_SPACES_SECRET_KEY: ${{ secrets.DIGITALOCEAN_SPACES_SECRET_KEY }}
+    steps:
+      - name: Checkout repo
+        uses: actions/checkout@v4
+        with:
+          clean: false
+          submodules: "recursive"
+
+      - name: Cache dependencies
+        uses: swatinem/rust-cache@v2
+        with:
+          save-if: ${{ github.ref == 'refs/heads/main' }}
+
+      - name: Configure linux
+        shell: bash -euxo pipefail {0}
+        run: script/linux
+
+      - name: Determine version and release channel
+        if: ${{ startsWith(github.ref, 'refs/tags/v') }}
+        run: |
+          set -eu
+
+          version=$(script/get-crate-version zed)
+          channel=$(cat crates/zed/RELEASE_CHANNEL)
+          echo "Publishing version: ${version} on release channel ${channel}"
+          echo "RELEASE_CHANNEL=${channel}" >> $GITHUB_ENV
+
+          expected_tag_name=""
+          case ${channel} in
+            stable)
+              expected_tag_name="v${version}";;
+            preview)
+              expected_tag_name="v${version}-pre";;
+            nightly)
+              expected_tag_name="v${version}-nightly";;
+            *)
+              echo "can't publish a release on channel ${channel}"
+              exit 1;;
+          esac
+          if [[ $GITHUB_REF_NAME != $expected_tag_name ]]; then
+            echo "invalid release tag ${GITHUB_REF_NAME}. expected ${expected_tag_name}"
+            exit 1
+          fi
+
+      # TODO linux : Find a way to add licenses to the final bundle
+      # - name: Generate license file
+      #   run: script/generate-licenses
+
+      - name: Create Linux *.deb bundle
+        run: script/bundle-linux
+
+      - name: Upload app bundle to workflow run if main branch or specific label
+        uses: actions/upload-artifact@v4
+        if: ${{ github.ref == 'refs/heads/main' }} || contains(github.event.pull_request.labels.*.name, 'run-bundling') }}
+        with:
+          name: Zed_${{ github.event.pull_request.head.sha || github.sha }}.deb
+          path: target/release/*.deb
+
+      # TODO linux : make it stable enough to be uploaded as a release
+      # - uses: softprops/action-gh-release@v1
+      #   name: Upload app bundle to release
+      #   if: ${{ env.RELEASE_CHANNEL == 'preview' || env.RELEASE_CHANNEL == 'stable' }}
+      #   with:
+      #     draft: true
+      #     prerelease: ${{ env.RELEASE_CHANNEL == 'preview' }}
+      #     files: target/release/Zed.dmg
+      #     body: ""
+      #   env:
+      #     GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/release_nightly.yml

@@ -50,8 +50,8 @@ jobs:
       - name: Run tests
         uses: ./.github/actions/run_tests
 
-  bundle:
-    name: Bundle app
+  bundle-mac:
+    name: Create a macOS bundle
     if: github.repository_owner == 'zed-industries'
     runs-on:
       - self-hosted
@@ -77,9 +77,6 @@ jobs:
           clean: false
           submodules: "recursive"
 
-      - name: Limit target directory size
-        run: script/clear-target-dir-if-larger-than 100
-
       - name: Set release channel to nightly
         run: |
           set -eu
@@ -90,8 +87,50 @@ jobs:
       - name: Generate license file
         run: script/generate-licenses
 
-      - name: Create app bundle
-        run: script/bundle
+      - name: Create macOS app bundle
+        run: script/bundle-mac
 
       - name: Upload Zed Nightly
-        run: script/upload-nightly
+        run: script/upload-nightly macos
+
+  bundle-deb:
+    name: Create a *.deb Linux bundle
+    if: github.repository_owner == 'zed-industries'
+    runs-on: ubuntu-22.04 # keep the version fixed to avoid libc and dynamic linked library issues
+    needs: tests
+    env:
+      DIGITALOCEAN_SPACES_ACCESS_KEY: ${{ secrets.DIGITALOCEAN_SPACES_ACCESS_KEY }}
+      DIGITALOCEAN_SPACES_SECRET_KEY: ${{ secrets.DIGITALOCEAN_SPACES_SECRET_KEY }}
+      ZED_CLIENT_CHECKSUM_SEED: ${{ secrets.ZED_CLIENT_CHECKSUM_SEED }}
+    steps:
+      - name: Checkout repo
+        uses: actions/checkout@v4
+        with:
+          clean: false
+          submodules: "recursive"
+
+      - name: Cache dependencies
+        uses: swatinem/rust-cache@v2
+        with:
+          save-if: ${{ github.ref == 'refs/heads/main' }}
+
+      - name: Configure linux
+        shell: bash -euxo pipefail {0}
+        run: script/linux
+
+      - name: Set release channel to nightly
+        run: |
+          set -euo pipefail
+          version=$(git rev-parse --short HEAD)
+          echo "Publishing version: ${version} on release channel nightly"
+          echo "nightly" > crates/zed/RELEASE_CHANNEL
+
+      # TODO linux : find a way to add licenses to the final bundle
+      # - name: Generate license file
+      #   run: script/generate-licenses
+
+      - name: Create Linux *.deb bundle
+        run: script/bundle-linux
+
+      - name: Upload Zed Nightly
+        run: script/upload-nightly linux-deb

crates/gpui/src/platform/linux/platform.rs

@@ -324,7 +324,7 @@ impl Platform for LinuxPlatform {
         })
     }
 
-    //todo!(linux)
+    //TODO linux
     fn app_path(&self) -> Result<PathBuf> {
         Err(anyhow::Error::msg(
             "Platform<LinuxPlatform>::app_path is not implemented yet",
@@ -338,7 +338,7 @@ impl Platform for LinuxPlatform {
         UtcOffset::UTC
     }
 
-    //todo!(linux)
+    //TODO linux
     fn path_for_auxiliary_executable(&self, name: &str) -> Result<PathBuf> {
         Err(anyhow::Error::msg(
             "Platform<LinuxPlatform>::path_for_auxiliary_executable is not implemented yet",
@@ -390,7 +390,7 @@ impl Platform for LinuxPlatform {
         })
     }
 
-    //todo!(linux): add trait methods for accessing the primary selection
+    //TODO linux: add trait methods for accessing the primary selection
 
     fn read_credentials(&self, url: &str) -> Task<Result<Option<(String, Vec<u8>)>>> {
         let url = url.to_string();

crates/gpui/src/platform/linux/wayland/client.rs

@@ -281,7 +281,7 @@ impl Client for WaylandClient {
             CursorStyle::ResizeUp => "n-resize".to_string(),
             CursorStyle::ResizeDown => "s-resize".to_string(),
             CursorStyle::ResizeUpDown => "ns-resize".to_string(),
-            CursorStyle::DisappearingItem => "grabbing".to_string(), // todo!(linux) - couldn't find equivalent icon in linux
+            CursorStyle::DisappearingItem => "grabbing".to_string(), // TODO linux - couldn't find equivalent icon in linux
             CursorStyle::IBeamCursorForVerticalLayout => "vertical-text".to_string(),
             CursorStyle::OperationNotAllowed => "not-allowed".to_string(),
             CursorStyle::DragLink => "dnd-link".to_string(),

crates/gpui/src/platform/linux/x11/client.rs

@@ -361,7 +361,7 @@ impl Client for X11Client {
         Box::new(X11Window(window_ptr))
     }
 
-    //todo!(linux)
+    // TODO linux
     fn set_cursor_style(&self, _style: CursorStyle) {}
 
     fn get_clipboard(&self) -> Rc<RefCell<dyn ClipboardProvider>> {

crates/zed/Cargo.toml

@@ -5,6 +5,7 @@ name = "zed"
 version = "0.127.0"
 publish = false
 license = "GPL-3.0-or-later"
+authors = ["Zed Team <hi@zed.dev>"]
 
 [lints]
 workspace = true

script/bundle-linux

@@ -0,0 +1,106 @@
+#!/usr/bin/env bash
+
+set -euxo pipefail
+
+build_flag="--release"
+target_dir="release"
+bundle_name=""
+zed_crate="zed"
+
+
+help_info() {
+  echo "
+Usage: ${0##*/} [options] [bundle_name]
+Build the application bundle for Linux.
+
+Options:
+  -d    Compile in debug mode
+  -h    Display this help and exit
+  "
+}
+
+while getopts 'dh' flag
+do
+    case "${flag}" in
+        d)
+            export CARGO_INCREMENTAL=true
+            export CARGO_BUNDLE_SKIP_BUILD=true
+            build_flag="";
+            target_dir="debug"
+            ;;
+        h)
+           help_info
+           exit 0
+           ;;
+    esac
+done
+
+shift $((OPTIND-1))
+
+if [[ $# -gt 0 ]]; then
+    if [ "$1" ]; then
+        bundle_name=$1
+    fi
+fi
+
+export ZED_BUNDLE=true
+
+cargo_bundle_version=$(cargo -q bundle --help 2>&1 | head -n 1 || echo "")
+if [ "$cargo_bundle_version" != "cargo-bundle v0.6.0-zed" ]; then
+    cargo install cargo-bundle --git https://github.com/zed-industries/cargo-bundle.git --branch zed-deploy
+fi
+
+echo "Compiling zed binaries"
+cargo build ${build_flag} --package ${zed_crate} --package cli
+
+echo "Creating application bundle"
+# TODO linux
+# Here, hacks to make `cargo bundle` run work, but macOS does not need these
+# Most probably, needs https://github.com/zed-industries/cargo-bundle/commit/9e185bd44d968d8039192220603494555afdbb4f from the upstream.
+cp "target/${target_dir}/Zed" "target/${target_dir}/zed"
+pushd crates/${zed_crate}
+    channel=$(<RELEASE_CHANNEL)
+    cp Cargo.toml Cargo.toml.backup
+    sed \
+        -i.backup -e \
+        "s/package.metadata.bundle-${channel}/package.metadata.bundle/" \
+        Cargo.toml
+
+    # TODO linux `zed_cli` does not get into this bundle despite being built
+    bundle_path=$(cargo bundle ${build_flag} --select-workspace-root | xargs)
+
+    mv Cargo.toml.backup Cargo.toml
+popd
+
+# For nightly, cut off the version out of the bundle name that `cargo bundle` always adds.
+if [ "$channel" == "nightly" ]; then
+    version="$(cargo metadata --no-deps --manifest-path crates/zed/Cargo.toml --offline --format-version=1 | jq -r '.packages | map(select(.name == "zed"))[0].version')"
+    version_less_bundle_path=$(echo "$bundle_path" | sed "s/_$version//")
+    mv "$bundle_path" "$version_less_bundle_path"
+    bundle_path="$version_less_bundle_path"
+fi
+
+# TODO linux
+# Other Linux systems will need a different set of manipulations + a way to know which ones to do.
+# If bundle_name is not set or empty, use the basename of $bundle_path
+if [ -z "${bundle_name}" ]; then
+    bundle_name=$(basename "${bundle_path}")
+fi
+# If bundle_name doesn't end in .deb, append it
+if [[ "$bundle_name" != *.deb ]]; then
+    bundle_name="$bundle_name.deb"
+fi
+
+pushd target/
+    rm -rf bundle/ 2>/dev/null || true
+    dpkg-deb -x "${bundle_path}" bundle/
+    dpkg-deb --control "${bundle_path}" bundle/DEBIAN
+    mkdir -p bundle/usr/local/bin/
+    mv bundle/usr/bin/zed "bundle/usr/local/bin/zed-$channel"
+    cp "${target_dir}/cli" "bundle/usr/local/bin/cli-$channel"
+    ln -s "/usr/local/bin/cli-$channel" "bundle/usr/local/bin/zed"
+    rm -rf bundle/usr/bin/
+    dpkg-deb -b bundle/ "${target_dir}/${bundle_name}"
+    bundle_path="${PWD}/${target_dir}/${bundle_name}"
+popd
+echo "Bundled ${bundle_path}"

script/bundle-mac

@@ -1,6 +1,6 @@
-#!/bin/bash
+#!/usr/bin/env bash
 
-set -e
+set -euxo pipefail
 
 build_flag="--release"
 target_dir="release"
@@ -19,7 +19,7 @@ APPLE_NOTORIZATION_TEAM="MQ55VZLNZQ"
 help_info() {
   echo "
 Usage: ${0##*/} [options] [bundle_name]
-Build the application bundle.
+Build the application bundle for macOS.
 
 Options:
   -d    Compile in debug mode
@@ -81,8 +81,10 @@ done
 
 shift $((OPTIND-1))
 
-if [ "$1" ]; then
-    bundle_name=$1
+if [[ $# -gt 0 ]]; then
+    if [ "$1" ]; then
+        bundle_name=$1
+    fi
 fi
 
 export ZED_BUNDLE=true
@@ -93,8 +95,6 @@ if [ "$cargo_bundle_version" != "cargo-bundle v0.6.0-zed" ]; then
     cargo install cargo-bundle --git https://github.com/zed-industries/cargo-bundle.git --branch zed-deploy
 fi
 
-rustup target add wasm32-wasi
-
 # Deal with versions of macOS that don't include libstdc++ headers
 export CXXFLAGS="-stdlib=libc++"
 
@@ -208,7 +208,7 @@ else
     echo "====== WARNING ======"
 
     # NOTE: if you need to test universal links you have a few paths forward:
-    # - create a PR and tag it with the `run-build-dmg` label, and download the .dmg file from there.
+    # - create a PR and tag it with the `run-bundling` label, and download the .dmg file from there.
     # - get a signing key for the MQ55VZLNZQ team from Nathan.
     # - create your own signing key, and update references to MQ55VZLNZQ to your own team ID
     # then comment out this line.

script/upload-nightly

@@ -1,7 +1,32 @@
-#!/bin/bash
+#!/usr/bin/env bash
 
 # Based on the template in: https://docs.digitalocean.com/reference/api/spaces-api/
-set -ux
+bash -euo pipefail
+
+allowed_targets=("linux-deb" "macos")
+is_allowed_target() {
+    for val in "${allowed_targets[@]}"; do
+        if [[ "$1" == "$val" ]]; then
+            return 0
+        fi
+    done
+    return 1
+}
+
+if [[ -n "${1:-}" ]]; then
+    if is_allowed_target "$1"; then
+        target="$1"
+    else
+        echo "Error: Target '$1' is not allowed"
+        echo "Usage: $0 [${allowed_targets[@]}]"
+        exit 1
+    fi
+else
+echo "Error: Target is not specified"
+echo "Usage: $0 [${allowed_targets[@]}]"
+exit 1
+fi
+echo "Uploading nightly for target: $target"
 
 # Step 1: Define the parameters for the Space you want to upload to.
 SPACE="zed-nightly-host" # Find your endpoint in the control panel, under Settings.
@@ -32,6 +57,19 @@ function uploadToSpaces
 
 sha=$(git rev-parse HEAD)
 echo ${sha} > target/latest-sha
-
-uploadToSpaces "target/release/Zed.dmg" "Zed.dmg"
-uploadToSpaces "target/latest-sha" "latest-sha"
+case "$target" in
+    macos)
+        uploadToSpaces "target/release/Zed.dmg" "Zed.dmg"
+        uploadToSpaces "target/latest-sha" "latest-sha"
+        ;;
+    linux-deb)
+        find target/release -type f -name "*.deb" -print0 | while IFS= read -r -d '' bundle_file; do
+            uploadToSpaces "$bundle_file" "$(basename "$bundle_file")"
+        done
+        uploadToSpaces "target/latest-sha" "latest-sha-linux-deb"
+        ;;
+    *)
+        echo "Error: Unknown target '$target'"
+        exit 1
+        ;;
+esac