Pull request: all: imp scripts

Merge in DNS/adguard-home from imp-sh to master

Squashed commit of the following:

commit 477832e11eca2ef7ac0071b5da938dacb2ed617d
Author: Ainar Garipov <A.Garipov@AdGuard.COM>
Date:   Wed May 19 20:24:34 2021 +0300

    scripts: rm dbg

commit dbb4b8c783f607781b980dcd57d78085c9022e72
Author: Ainar Garipov <A.Garipov@AdGuard.COM>
Date:   Wed May 19 20:21:20 2021 +0300

    all: imp code, compat, docs

commit e6e4375d67ad1c213efb04411e3ba0bc6293f936
Author: Ainar Garipov <A.Garipov@AdGuard.COM>
Date:   Wed May 19 19:33:48 2021 +0300

    all: imp scripts
This commit is contained in:
Ainar Garipov 2021-05-19 20:31:20 +03:00
parent b3d28408cd
commit 6f7fd33afd
17 changed files with 473 additions and 321 deletions

View File

@ -46,7 +46,7 @@
'restore-keys': '${{ runner.os }}-go-'
- 'name': 'Get npm cache directory'
'id': 'npm-cache'
'run': 'echo "::set-output name=dir::$(npm config get cache)"'
'run': 'echo "::set-output name=dir::$( npm config get cache )"'
- 'name': 'Set up npm cache'
'uses': 'actions/cache@v2'
'with':

View File

@ -1,9 +1,9 @@
# AdGuard Home Developer Guidelines
As of **March 2021**, following this document is obligatory for all new code.
Some of the rules aren't enforced as thoroughly or remain broken in old code,
but this is still the place to find out about what we **want** our code to look
like and how to improve it.
Following this document is obligatory for all new code. Some of the rules
aren't enforced as thoroughly or remain broken in old code, but this is still
the place to find out about what we **want** our code to look like and how to
improve it.
The rules are mostly sorted in the alphabetical order.
@ -21,6 +21,7 @@ The rules are mostly sorted in the alphabetical order.
* [Recommended Reading](#recommended-reading)
* [Markdown](#markdown)
* [Shell Scripting](#shell-scripting)
* [Shell Conditionals](#shell-cond)
* [Text, Including Comments](#text-including-comments)
* [YAML](#yaml)
@ -315,6 +316,26 @@ on GitHub and most other Markdown renderers. -->
* Avoid spaces between patterns of the same `case` condition.
* `export` and `readonly` should be used separately from variable assignment,
because otherwise failures in command substitutions won't stop the script.
That is, do this:
```sh
X="$( echo 42 )"
export X
```
And **not** this:
```sh
# Bad!
export X="$( echo 42 )"
```
* If a binary value is needed, use `0` for `false`, and `1` for `true`.
* Mark every variable that shouldn't change later as `readonly`.
* Prefer `'raw strings'` to `"double quoted strings"` whenever possible.
* Put spaces within `$( cmd )`, `$(( expr ))`, and `{ cmd; }`.
@ -330,8 +351,6 @@ on GitHub and most other Markdown renderers. -->
* UPPERCASE names for external exported variables, lowercase for local,
unexported ones.
* Use `readonly` liberally.
* Use `set -e -f -u` and also `set -x` in verbose mode.
* Use the `"$var"` form instead of the `$var` form, unless word splitting is
@ -355,14 +374,23 @@ on GitHub and most other Markdown renderers. -->
dir="${TOP_DIR}"/sub
```
* When using `test` (aka `[`), spell compound conditions with `&&`, `||`, and
`!` **outside** of `test` instead of `-a`, `-o`, and `!` inside of `test`
correspondingly. The latter ones are pretty much deprecated in POSIX.
Also, prefer `!= ''` form instead of `-n` to check if string is empty.
### <a id="shell-cond" href="#shell-cond">Shell Conditionals</a>
Guidelines and agreements for using command `test`, also known as `[`:
* Prefer the `!= ''` form instead of using `-n` to check if string is empty.
* Spell compound conditions with `&&`, `||`, and `!` **outside** of `test`
instead of `-a`, `-o`, and `!` **inside** of `test` correspondingly. The
latter ones are pretty much deprecated in POSIX.
See also: “[Problems With the `test` Builtin: What Does `-a` Mean?][test]”.
* Use `=` for strings and `-eq` for numbers to catch typing errors.
[test]: https://www.oilshell.org/blog/2017/08/31.html
See also: “[Problems With the `test` Builtin: What Does `-a` Mean?]”.
[Problems With the `test` Builtin: What Does `-a` Mean?]: https://www.oilshell.org/blog/2017/08/31.html
## <a id="text-including-comments" href="#text-including-comments">Text, Including Comments</a>
@ -400,6 +428,8 @@ on GitHub and most other Markdown renderers. -->
// TODO(usr1, usr2): Fix the frobulation issue.
```
## <a id="yaml" href="#yaml">YAML</a>
* **TODO(a.garipov):** Define naming conventions for schema names in our

View File

@ -106,3 +106,5 @@ go-os-check:
openapi-lint: ; cd ./openapi/ && $(YARN) test
openapi-show: ; cd ./openapi/ && $(YARN) start
txt-lint: ; $(ENV) "$(SHELL)" ./scripts/make/txt-lint.sh

View File

@ -152,7 +152,9 @@
cd ./dist/
export CHANNEL="${bamboo.channel}"
CHANNEL="${bamboo.channel}"
export CHANNEL
../bamboo-deploy-publisher/deploy.sh adguard-home-"$CHANNEL"
'final-tasks':
- 'clean'

View File

@ -2,22 +2,27 @@
set -e -f -u
if [ "$(git diff --cached --name-only -- 'client/*.js')" ]
if [ "$( git diff --cached --name-only -- 'client/*.js' )" ]
then
make js-lint js-test
fi
if [ "$(git diff --cached --name-only -- 'client2/*.js' 'client2/*.ts' 'client2/*.tsx')" ]
if [ "$( git diff --cached --name-only -- 'client2/*.js' 'client2/*.ts' 'client2/*.tsx' )" ]
then
make js-beta-lint js-beta-test
fi
if [ "$(git diff --cached --name-only -- '*.go' 'go.mod')" ]
if [ "$( git diff --cached --name-only -- '*.go' '*.mod' '*.sh' 'Makefile' )" ]
then
make go-os-check go-lint go-test
fi
if [ "$(git diff --cached --name-only -- './openapi/openapi.yaml')" ]
if [ "$( git diff --cached --name-only -- '*.md' '*.yaml' '*.yml' )" ]
then
make txt-lint
fi
if [ "$( git diff --cached --name-only -- './openapi/openapi.yaml' )" ]
then
make openapi-lint
fi

View File

@ -46,8 +46,9 @@ is_little_endian() {
# unzip (macOS) / tar (other unices)
#
check_required() {
readonly required_darwin="unzip"
readonly required_unix="tar"
required_darwin="unzip"
required_unix="tar"
readonly required_darwin required_unix
# Split with space.
required="curl"
@ -132,7 +133,7 @@ parse_opts() {
esac
done
if [ "$uninstall" = '1' ] && [ "$reinstall" = '1' ]
if [ "$uninstall" -eq '1' ] && [ "$reinstall" -eq '1' ]
then
error_exit 'the -r and -u options are mutually exclusive'
fi
@ -277,7 +278,9 @@ fix_freebsd() {
return 0
fi
readonly rcd='/usr/local/etc/rc.d'
rcd='/usr/local/etc/rc.d'
readonly rcd
if ! [ -d "$rcd" ]
then
mkdir "$rcd"
@ -292,16 +295,17 @@ configure() {
fix_darwin
check_out_dir
readonly pkg_name="AdGuardHome_${os}_${cpu}.${pkg_ext}"
readonly url="https://static.adguard.com/adguardhome/${channel}/${pkg_name}"
readonly agh_dir="${out_dir}/AdGuardHome"
pkg_name="AdGuardHome_${os}_${cpu}.${pkg_ext}"
url="https://static.adguard.com/adguardhome/${channel}/${pkg_name}"
agh_dir="${out_dir}/AdGuardHome"
readonly pkg_name url agh_dir
log "AdGuard Home will be installed into $agh_dir"
}
# Function is_root checks for root privileges to be granted.
is_root() {
if [ "$( id -u )" = '0' ]
if [ "$( id -u )" -eq '0' ]
then
log 'script is executed with root privileges'
@ -325,24 +329,26 @@ please, restart it with root privileges'
#
# TODO(e.burkov): Try to avoid restarting.
rerun_with_root() {
readonly script_url=\
script_url=\
'https://raw.githubusercontent.com/AdguardTeam/AdGuardHome/master/scripts/install.sh'
readonly script_url
flags=''
if [ "$reinstall" = '1' ]
if [ "$reinstall" -eq '1' ]
then
flags="${flags} -r"
fi
if [ "$uninstall" = '1' ]
if [ "$uninstall" -eq '1' ]
then
flags="${flags} -u"
fi
if [ "$verbose" = '1' ]
if [ "$verbose" -eq '1' ]
then
flags="${flags} -v"
fi
readonly opts="-c $channel -C $cpu -O $os -o $out_dir $flags"
opts="-c $channel -C $cpu -O $os -o $out_dir $flags"
readonly opts
log 'restarting with root privileges'
@ -397,7 +403,7 @@ handle_existing() {
then
log 'no need to uninstall'
if [ "$uninstall" = '1' ]
if [ "$uninstall" -eq '1' ]
then
exit 0
fi
@ -428,7 +434,7 @@ handle_existing() {
log 'AdGuard Home was successfully uninstalled'
fi
if [ "$uninstall" = '1' ]
if [ "$uninstall" -eq '1' ]
then
exit 0
fi

View File

@ -14,58 +14,65 @@ fi
set -e -f -u
# Require these to be set. The channel value is validated later.
readonly channel="$CHANNEL"
readonly commit="$COMMIT"
readonly dist_dir="$DIST_DIR"
channel="$CHANNEL"
commit="$COMMIT"
dist_dir="$DIST_DIR"
readonly channel commit dist_dir
if [ "${VERSION:-}" = 'v0.0.0' ] || [ "${VERSION:-}" = '' ]
then
readonly version="$(sh ./scripts/make/version.sh)"
version="$( sh ./scripts/make/version.sh )"
else
readonly version="$VERSION"
version="$VERSION"
fi
readonly version
echo $version
# Allow users to use sudo.
readonly sudo_cmd="${SUDO:-}"
sudo_cmd="${SUDO:-}"
readonly sudo_cmd
readonly docker_platforms="\
docker_platforms="\
linux/386,\
linux/amd64,\
linux/arm/v6,\
linux/arm/v7,\
linux/arm64,\
linux/ppc64le"
readonly docker_platforms
readonly build_date="$(date -u +'%Y-%m-%dT%H:%M:%SZ')"
build_date="$( date -u +'%Y-%m-%dT%H:%M:%SZ' )"
readonly build_date
# Set DOCKER_IMAGE_NAME to 'adguard/adguard-home' if you want (and are
# allowed) to push to DockerHub.
readonly docker_image_name="${DOCKER_IMAGE_NAME:-adguardhome-dev}"
# Set DOCKER_IMAGE_NAME to 'adguard/adguard-home' if you want (and are allowed)
# to push to DockerHub.
docker_image_name="${DOCKER_IMAGE_NAME:-adguardhome-dev}"
readonly docker_image_name
# Set DOCKER_OUTPUT to 'type=image,name=adguard/adguard-home,push=true'
# if you want (and are allowed) to push to DockerHub.
readonly docker_output="${DOCKER_OUTPUT:-type=image,name=${docker_image_name},push=false}"
# Set DOCKER_OUTPUT to 'type=image,name=adguard/adguard-home,push=true' if you
# want (and are allowed) to push to DockerHub.
docker_output="${DOCKER_OUTPUT:-type=image,name=${docker_image_name},push=false}"
readonly docker_output
case "$channel"
in
('release')
readonly docker_image_full_name="${docker_image_name}:${version}"
readonly docker_tags="--tag ${docker_image_name}:latest"
docker_image_full_name="${docker_image_name}:${version}"
docker_tags="--tag ${docker_image_name}:latest"
;;
('beta')
readonly docker_image_full_name="${docker_image_name}:${version}"
readonly docker_tags="--tag ${docker_image_name}:beta"
docker_image_full_name="${docker_image_name}:${version}"
docker_tags="--tag ${docker_image_name}:beta"
;;
('edge')
# Don't set the version tag when pushing to the edge channel.
readonly docker_image_full_name="${docker_image_name}:edge"
readonly docker_tags=''
docker_image_full_name="${docker_image_name}:edge"
docker_tags=''
;;
('development')
readonly docker_image_full_name="${docker_image_name}"
readonly docker_tags=''
docker_image_full_name="${docker_image_name}"
docker_tags=''
;;
(*)
echo "invalid channel '$channel', supported values are\
@ -73,11 +80,14 @@ in
exit 1
;;
esac
readonly docker_image_full_name docker_tags
# Copy the binaries into a new directory under new names, so that it's eaiser to
# COPY them later. DO NOT remove the trailing underscores. See file
# scripts/make/Dockerfile.
dist_docker="${dist_dir}/docker"
readonly dist_docker
# Copy the binaries into a new directory under new names, so that it's
# eaiser to COPY them later. DO NOT remove the trailing underscores.
# See scripts/make/Dockerfile.
readonly dist_docker="${dist_dir}/docker"
mkdir -p "$dist_docker"
cp "${dist_dir}/AdGuardHome_linux_386/AdGuardHome/AdGuardHome"\
"${dist_docker}/AdGuardHome_linux_386_"
@ -92,8 +102,8 @@ cp "${dist_dir}/AdGuardHome_linux_arm_7/AdGuardHome/AdGuardHome"\
cp "${dist_dir}/AdGuardHome_linux_ppc64le/AdGuardHome/AdGuardHome"\
"${dist_docker}/AdGuardHome_linux_ppc64le_"
# Don't use quotes with $docker_tags and $debug_flags because we want
# word splitting and or an empty space if tags are empty.
# Don't use quotes with $docker_tags and $debug_flags because we want word
# splitting and or an empty space if tags are empty.
$sudo_cmd docker\
$debug_flags\
buildx build\

View File

@ -2,18 +2,20 @@
# AdGuard Home Release Script
#
# The commentary in this file is written with the assumption that the
# reader only has superficial knowledge of the POSIX shell language and
# alike. Experienced readers may find it overly verbose.
# The commentary in this file is written with the assumption that the reader
# only has superficial knowledge of the POSIX shell language and alike.
# Experienced readers may find it overly verbose.
# The default verbosity level is 0. Show log messages if the caller
# requested verbosity level greather than 0. Show every command that is
# run if the verbosity level is greater than 1. Show the environment if
# the verbosity level is greater than 2. Otherwise, print nothing.
# The default verbosity level is 0. Show log messages if the caller requested
# verbosity level greather than 0. Show every command that is run if the
# verbosity level is greater than 1. Show the environment if the verbosity
# level is greater than 2. Otherwise, print nothing.
#
# The level of verbosity for the build script is the same minus one
# level. See below in build().
readonly verbose="${VERBOSE:-0}"
# The level of verbosity for the build script is the same minus one level. See
# below in build().
verbose="${VERBOSE:-0}"
readonly verbose
if [ "$verbose" -gt '2' ]
then
env
@ -24,14 +26,15 @@ then
fi
# By default, sign the packages, but allow users to skip that step.
readonly sign="${SIGN:-1}"
sign="${SIGN:-1}"
readonly sign
# Exit the script if a pipeline fails (-e), prevent accidental filename
# expansion (-f), and consider undefined variables as errors (-u).
set -e -f -u
# Function log is an echo wrapper that writes to stderr if the caller
# requested verbosity level greater than 0. Otherwise, it does nothing.
# Function log is an echo wrapper that writes to stderr if the caller requested
# verbosity level greater than 0. Otherwise, it does nothing.
log() {
if [ "$verbose" -gt '0' ]
then
@ -42,85 +45,113 @@ log() {
log 'starting to build AdGuard Home release'
# Require the channel to be set. Additional validation is performed
# later by go-build.sh.
readonly channel="$CHANNEL"
# Require the channel to be set. Additional validation is performed later by
# go-build.sh.
channel="$CHANNEL"
readonly channel
# Check VERSION against the default value from the Makefile. If it is
# that, use the version calculation script.
# Check VERSION against the default value from the Makefile. If it is that, use
# the version calculation script.
if [ "${VERSION:-}" = 'v0.0.0' ] || [ "${VERSION:-}" = '' ]
then
readonly version="$( sh ./scripts/make/version.sh )"
version="$( sh ./scripts/make/version.sh )"
else
readonly version="$VERSION"
version="$VERSION"
fi
readonly version
log "channel '$channel'"
log "version '$version'"
# Check architecture and OS limiters. Add spaces to the local versions
# for better pattern matching.
# Check architecture and OS limiters. Add spaces to the local versions for
# better pattern matching.
if [ "${ARCH:-}" != '' ]
then
log "arches: '$ARCH'"
readonly arches=" $ARCH "
arches=" $ARCH "
else
readonly arches=''
arches=''
fi
readonly arches
if [ "${OS:-}" != '' ]
then
log "oses: '$OS'"
readonly oses=" $OS "
oses=" $OS "
else
readonly oses=''
oses=''
fi
readonly oses
readonly snap_enabled="${SNAP:-1}"
if [ "$snap_enabled" = '0' ]
snap_enabled="${SNAP:-1}"
readonly snap_enabled
if [ "$snap_enabled" -eq '0' ]
then
log 'snap: disabled'
fi
# Require the gpg key and passphrase to be set if the signing is
# required.
if [ "$sign" = '1' ]
# Require the gpg key and passphrase to be set if the signing is required.
if [ "$sign" -eq '1' ]
then
readonly gpg_key_passphrase="$GPG_KEY_PASSPHRASE"
readonly gpg_key="$GPG_KEY"
gpg_key_passphrase="$GPG_KEY_PASSPHRASE"
gpg_key="$GPG_KEY"
else
gpg_key_passphrase=''
gpg_key=''
fi
readonly gpg_key_passphrase gpg_key
# The default distribution files directory is dist.
readonly dist="${DIST_DIR:-dist}"
dist="${DIST_DIR:-dist}"
readonly dist
# Give users the ability to override the go command from environment.
# For example, to build two releases with two different Go versions and
# test the difference.
readonly go="${GO:-go}"
# Give users the ability to override the go command from environment. For
# example, to build two releases with two different Go versions and test the
# difference.
go="${GO:-go}"
readonly go
log "checking tools"
# Make sure we fail gracefully if one of the tools we need is missing.
for tool in gpg gzip sed sha256sum snapcraft tar zip
# Make sure we fail gracefully if one of the tools we need is missing. Use
# alternatives when available.
sha256sum_cmd='sha256sum'
for tool in gpg gzip sed "$sha256sum_cmd" snapcraft tar zip
do
which "$tool" >/dev/null\
|| { log "pieces don't fit, '$tool' not found"; exit 1; }
if ! which "$tool" > /dev/null
then
if [ "$tool" = "$sha256sum_cmd" ] && which 'shasum' > /dev/null
then
# macOS doesn't have sha256sum installed by default, but
# it does have shasum.
log 'replacing sha256sum with shasum -a 256'
sha256sum_cmd='shasum -a 256'
else
log "pieces don't fit, '$tool' not found"
exit 1
fi
fi
done
readonly sha256sum_cmd
# Data section. Arrange data into space-separated tables for read -r to
# read. Use 0 for missing values.
# Data section. Arrange data into space-separated tables for read -r to read.
# Use 0 for missing values.
readonly arms='5
arms='5
6
7'
readonly arms
readonly mipses='softfloat'
mipses='softfloat'
readonly mipses
# TODO(a.garipov): Remove armv6, because it was always overwritten by
# armv7. Rename armv7 to armhf. Rename the 386 snap to i386.
# TODO(a.garipov): Remove armv6, because it was always overwritten by armv7.
# Rename armv7 to armhf. Rename the 386 snap to i386.
# os arch arm mips snap
readonly platforms="\
platforms="\
darwin amd64 0 0 0
freebsd 386 0 0 0
freebsd amd64 0 0 0
@ -141,12 +172,13 @@ linux mipsle 0 softfloat 0
linux ppc64le 0 0 0
windows 386 0 0 0
windows amd64 0 0 0"
readonly platforms
# Function build builds the release for one platform. It builds
# a binary, an archive and, if needed, a snap package.
# Function build builds the release for one platform. It builds a binary, an
# archive and, if needed, a snap package.
build() {
# Get the arguments. Here and below, use the "build_" prefix
# for all variables local to function build.
# Get the arguments. Here and below, use the "build_" prefix for all
# variables local to function build.
build_dir="${dist}/${1}/AdGuardHome"\
build_ar="$2"\
build_os="$3"\
@ -156,8 +188,7 @@ build() {
build_snap="$7"\
;
# Use the ".exe" filename extension if we build a Windows
# release.
# Use the ".exe" filename extension if we build a Windows release.
if [ "$build_os" = 'windows' ]
then
build_output="./${build_dir}/AdGuardHome.exe"
@ -169,11 +200,11 @@ build() {
# Build the binary.
#
# Set GOARM and GOMIPS to an empty string if $build_arm and
# $build_mips are zero by removing the zero as if it's a prefix.
# Set GOARM and GOMIPS to an empty string if $build_arm and $build_mips
# are zero by removing the zero as if it's a prefix.
#
# Don't use quotes with $build_par because we want an empty
# space if parallelism wasn't set.
# Don't use quotes with $build_par because we want an empty space if
# parallelism wasn't set.
env\
GOARCH="$build_arch"\
GOARM="${build_arm#0}"\
@ -187,7 +218,7 @@ build() {
log "$build_output"
if [ "$sign" = '1' ]
if [ "$sign" -eq '1' ]
then
gpg\
--default-key "$gpg_key"\
@ -202,8 +233,8 @@ build() {
# Prepare the build directory for archiving.
cp ./CHANGELOG.md ./LICENSE.txt ./README.md "$build_dir"
# Make archives. Windows and macOS prefer ZIP archives; the
# rest, gzipped tarballs.
# Make archives. Windows and macOS prefer ZIP archives; the rest,
# gzipped tarballs.
case "$build_os"
in
('darwin'|'windows')
@ -220,7 +251,11 @@ build() {
log "$build_archive"
if [ "$build_snap" = '0' ] || [ "$snap_enabled" = '0' ]
# build_snap is a string, so use string comparison for it.
#
# TODO(a.garipov): Consider using a different empty value in the
# platforms table.
if [ "$build_snap" = '0' ] || [ "$snap_enabled" -eq '0' ]
then
return
fi
@ -257,10 +292,10 @@ build() {
./scripts/snap/snap.tmpl.yaml\
>"${build_snap_dir}/meta/snap.yaml"
# TODO(a.garipov): The snapcraft tool will *always* write
# everything, including errors, to stdout. And there doesn't
# seem to be a way to change that. So, save the combined
# output, but only show it when snapcraft actually fails.
# TODO(a.garipov): The snapcraft tool will *always* write everything,
# including errors, to stdout. And there doesn't seem to be a way to
# change that. So, save the combined output, but only show it when
# snapcraft actually fails.
set +e
build_snapcraft_output="$(
snapcraft pack "$build_snap_dir" --output "$build_snap_output" 2>&1
@ -278,20 +313,18 @@ build() {
log "starting builds"
# Go over all platforms defined in the space-separated table above,
# tweak the values where necessary, and feed to build.
# Go over all platforms defined in the space-separated table above, tweak the
# values where necessary, and feed to build.
echo "$platforms" | while read -r os arch arm mips snap
do
# See if the architecture or the OS is in the allowlist. To do
# so, try removing everything that matches the pattern (well,
# a prefix, but that doesn't matter here) containing the arch or
# the OS.
# See if the architecture or the OS is in the allowlist. To do so, try
# removing everything that matches the pattern (well, a prefix, but that
# doesn't matter here) containing the arch or the OS.
#
# For example, when $arches is " amd64 arm64 " and $arch is
# "amd64", then the pattern to remove is "* amd64 *", so the
# whole string becomes empty. On the other hand, if $arch is
# "windows", then the pattern is "* windows *", which doesn't
# match, so nothing is removed.
# For example, when $arches is " amd64 arm64 " and $arch is "amd64",
# then the pattern to remove is "* amd64 *", so the whole string becomes
# empty. On the other hand, if $arch is "windows", then the pattern is
# "* windows *", which doesn't match, so nothing is removed.
#
# See https://stackoverflow.com/a/43912605/1892060.
if [ "${arches##* $arch *}" != '' ]
@ -333,17 +366,16 @@ log "$build_archive"
log "calculating checksums"
# Calculate the checksums of the files in a subshell with a different
# working directory. Don't use ls, because files matching one of the
# patterns may be absent, which will make ls return with a non-zero
# status code.
# Calculate the checksums of the files in a subshell with a different working
# directory. Don't use ls, because files matching one of the patterns may be
# absent, which will make ls return with a non-zero status code.
(
cd "./${dist}"
files="$( find . ! -name . -prune \( -name '*.tar.gz' -o -name '*.zip' \) )"
# Don't use quotes to get word splitting.
sha256sum $files > ./checksums.txt
$sha256sum_cmd $files > ./checksums.txt
)
log "writing versions"
@ -351,20 +383,19 @@ log "writing versions"
echo "version=$version" > "./${dist}/version.txt"
# Create the verison.json file.
#
# TODO(a.garipov): Perhaps rewrite this as a go run program. Dealing
# with structured documents is really not a Shell's job.
readonly version_download_url="https://static.adguard.com/adguardhome/${channel}"
readonly version_json="./${dist}/version.json"
version_download_url="https://static.adguard.com/adguardhome/${channel}"
version_json="./${dist}/version.json"
readonly version_download_url version_json
# Point users to the master branch if the channel is edge.
if [ "$channel" = 'edge' ]
then
readonly version_history_url='https://github.com/AdguardTeam/AdGuardHome/commits/master'
version_history_url='https://github.com/AdguardTeam/AdGuardHome/commits/master'
else
readonly version_history_url='https://github.com/AdguardTeam/AdGuardHome/releases'
version_history_url='https://github.com/AdguardTeam/AdGuardHome/releases'
fi
readonly version_history_url
rm -f "$version_json"
echo "{
@ -374,23 +405,11 @@ echo "{
\"selfupdate_min_version\": \"0.0\",
" >> "$version_json"
# Add the old object keys for compatibility with pre-v0.105.0 MIPS that
# did not mention the softfloat variant.
#
# TODO(a.garipov): Remove this around the time we hit v0.107.0.
echo "
\"download_linux_mips\": \"${version_download_url}/AdGuardHome_linux_mips_softfloat.tar.gz\",
\"download_linux_mipsle\": \"${version_download_url}/AdGuardHome_linux_mipsle_softfloat.tar.gz\",
\"download_linux_mips64\": \"${version_download_url}/AdGuardHome_linux_mips64_softfloat.tar.gz\",
\"download_linux_mips64le\": \"${version_download_url}/AdGuardHome_linux_mips64le_softfloat.tar.gz\",
" >> "$version_json"
# Same as with checksums above, don't use ls, because files matching one
# of the patterns may be absent.
readonly ar_files="$( \
find "./${dist}/" ! -name "${dist}" -prune \( -name '*.tar.gz' -o -name '*.zip' \)
)"
readonly ar_files_len="$( echo "$ar_files" | wc -l )"
# Same as with checksums above, don't use ls, because files matching one of the
# patterns may be absent.
ar_files="$( find "./${dist}/" ! -name "${dist}" -prune \( -name '*.tar.gz' -o -name '*.zip' \) )"
ar_files_len="$( echo "$ar_files" | wc -l )"
readonly ar_files ar_files_len
i='1'
# Don't use quotes to get word splitting.
@ -408,7 +427,7 @@ do
# Use the filename's base path.
filename="${f#./${dist}/}"
if [ "$i" = "$ar_files_len" ]
if [ "$i" -eq "$ar_files_len" ]
then
echo " \"download_${platform}\": \"${version_download_url}/${filename}\"" >> "$version_json"
else

View File

@ -12,9 +12,9 @@ set -e -f -u
dist_dir="$DIST_DIR"
go="${GO:-go}"
# Set the GOPATH explicitly in case make clean is called from under sudo
# after a Docker build.
env PATH="$("$go" env GOPATH)/bin":"$PATH" packr clean
# Set the GOPATH explicitly in case make clean is called from under sudo after
# a Docker build.
env PATH="$( "$go" env GOPATH )/bin":"$PATH" packr clean
rm -f\
./AdGuardHome\

View File

@ -2,41 +2,47 @@
# AdGuard Home Build Script
#
# The commentary in this file is written with the assumption that the
# reader only has superficial knowledge of the POSIX shell language and
# alike. Experienced readers may find it overly verbose.
# The commentary in this file is written with the assumption that the reader
# only has superficial knowledge of the POSIX shell language and alike.
# Experienced readers may find it overly verbose.
# The default verbosity level is 0. Show every command that is run and every
# package that is processed if the caller requested verbosity level greater than
# 0. Also show subcommands if the requested verbosity level is greater than 1.
# Otherwise, do nothing.
verbose="${VERBOSE:-0}"
readonly verbose
# The default verbosity level is 0. Show every command that is run and
# every package that is processed if the caller requested verbosity
# level greater than 0. Also show subcommands if the requested
# verbosity level is greater than 1. Otherwise, do nothing.
readonly verbose="${VERBOSE:-0}"
if [ "$verbose" -gt '1' ]
then
env
set -x
readonly v_flags='-v'
readonly x_flags='-x'
v_flags='-v'
x_flags='-x'
elif [ "$verbose" -gt '0' ]
then
set -x
readonly v_flags='-v'
readonly x_flags=''
v_flags='-v'
x_flags=''
else
set +x
readonly v_flags=''
readonly x_flags=''
v_flags=''
x_flags=''
fi
readonly x_flags v_flags
# Exit the script if a pipeline fails (-e), prevent accidental filename
# expansion (-f), and consider undefined variables as errors (-u).
set -e -f -u
# Allow users to set the Go version.
readonly go="${GO:-go}"
go="${GO:-go}"
readonly go
# Require the channel to be set and validate the value.
readonly channel="$CHANNEL"
channel="$CHANNEL"
readonly channel
case "$channel"
in
('development'|'edge'|'beta'|'release')
@ -52,15 +58,19 @@ esac
# Require the version to be set.
#
# TODO(a.garipov): Additional validation?
readonly version="$VERSION"
version="$VERSION"
readonly version
# Set date and time of the current build unless already set.
readonly buildtime="${BUILD_TIME:-$( date -u +%FT%TZ%z )}"
buildtime="${BUILD_TIME:-$( date -u +%FT%TZ%z )}"
readonly buildtime
# Set the linker flags accordingly: set the release channel and the current
# version as well as goarm and gomips variable values, if the variables are set
# and are not empty.
version_pkg='github.com/AdguardTeam/AdGuardHome/internal/version'
readonly version_pkg
# Set the linker flags accordingly: set the release channel and the
# current version as well as goarm and gomips variable values, if the
# variables are set and are not empty.
readonly version_pkg='github.com/AdguardTeam/AdGuardHome/internal/version'
ldflags="-s -w"
ldflags="${ldflags} -X ${version_pkg}.version=${version}"
ldflags="${ldflags} -X ${version_pkg}.channel=${channel}"
@ -74,39 +84,47 @@ then
fi
# Allow users to limit the build's parallelism.
readonly parallelism="${PARALLELISM:-}"
if [ "$parallelism" != '' ]
parallelism="${PARALLELISM:-}"
readonly parallelism
if [ "${parallelism}" != '' ]
then
readonly par_flags="-p ${parallelism}"
par_flags="-p ${parallelism}"
else
readonly par_flags=''
par_flags=''
fi
readonly par_flags
# Allow users to specify a different output name.
readonly out="${OUT:-}"
out="${OUT:-}"
readonly out
if [ "$out" != '' ]
then
readonly out_flags="-o ${out}"
out_flags="-o ${out}"
else
readonly out_flags=''
out_flags=''
fi
readonly out_flags
# Allow users to enable the race detector. Unfortunately, that means
# that CGo must be enabled.
readonly race="${RACE:-0}"
if [ "$race" = '0' ]
# Allow users to enable the race detector. Unfortunately, that means that cgo
# must be enabled.
if [ "${RACE:-0}" -eq '0' ]
then
readonly cgo_enabled='0'
readonly race_flags=''
cgo_enabled='0'
race_flags=''
else
readonly cgo_enabled='1'
readonly race_flags='--race'
cgo_enabled='1'
race_flags='--race'
fi
readonly cgo_enabled race_flags
export CGO_ENABLED="$cgo_enabled"
export GO111MODULE='on'
CGO_ENABLED="$cgo_enabled"
GO111MODULE='on'
export CGO_ENABLED GO111MODULE
readonly build_flags="${BUILD_FLAGS:-$race_flags --trimpath $out_flags $par_flags $v_flags $x_flags}"
build_flags="${BUILD_FLAGS:-$race_flags --trimpath $out_flags $par_flags $v_flags $x_flags}"
readonly build_flags
# Don't use quotes with flag variables to get word splitting.
"$go" generate $v_flags $x_flags ./main.go

View File

@ -1,36 +1,37 @@
#!/bin/sh
verbose="${VERBOSE:-0}"
readonly verbose
if [ "$verbose" -gt '1' ]
then
env
set -x
readonly v_flags='-v'
readonly x_flags='-x'
v_flags='-v'
x_flags='-x'
elif [ "$verbose" -gt '0' ]
then
set -x
readonly v_flags='-v'
readonly x_flags=''
v_flags='-v'
x_flags=''
else
set +x
readonly v_flags=''
readonly x_flags=''
v_flags=''
x_flags=''
fi
readonly v_flags x_flags
set -e -f -u
go="${GO:-go}"
# Don't use quotes with flag variables because we want an empty space if
# those aren't set.
# Don't use quotes with flag variables because we want an empty space if those
# aren't set.
"$go" mod download $x_flags
# Reset GOARCH and GOOS to make sure we install the tools for the native
# architecture even when we're cross-compiling the main binary, and also
# to prevent the "cannot install cross-compiled binaries when GOBIN is
# set" error.
# architecture even when we're cross-compiling the main binary, and also to
# prevent the "cannot install cross-compiled binaries when GOBIN is set" error.
env\
GOARCH=""\
GOOS=""\

View File

@ -2,44 +2,42 @@
verbose="${VERBOSE:-0}"
# Verbosity levels:
# 0 = Don't print anything except for errors.
# 1 = Print commands, but not nested commands.
# 2 = Print everything.
# Set verbosity.
if [ "$verbose" -gt '0' ]
then
set -x
fi
# Set $EXIT_ON_ERROR to zero to see all errors.
if [ "${EXIT_ON_ERROR:-1}" = '0' ]
if [ "${EXIT_ON_ERROR:-1}" -eq '0' ]
then
set +e
else
set -e
fi
# We don't need glob expansions and we want to see errors about unset
# variables.
# We don't need glob expansions and we want to see errors about unset variables.
set -f -u
# Deferred Helpers
readonly not_found_msg='
not_found_msg='
looks like a binary not found error.
make sure you have installed the linter binaries using:
$ make go-tools
'
readonly not_found_msg
# TODO(a.garipov): Put it into a separate script and source it both here and in
# txt-lint.sh?
not_found() {
if [ "$?" = '127' ]
if [ "$?" -eq '127' ]
then
# Code 127 is the exit status a shell uses when
# a command or a file is not found, according to the
# Bash Hackers wiki.
# Code 127 is the exit status a shell uses when a command or
# a file is not found, according to the Bash Hackers wiki.
#
# See https://wiki.bash-hackers.org/dict/terms/exit_status.
echo "$not_found_msg" 1>&2
@ -51,15 +49,17 @@ trap not_found EXIT
# Warnings
readonly go_min_version='go1.15'
readonly go_min_version_prefix="go version ${go_min_version}"
readonly go_version_msg="
go_min_version='go1.15'
go_min_version_prefix="go version ${go_min_version}"
go_version_msg="
warning: your go version is different from the recommended minimal one (${go_min_version}).
if you have the version installed, please set the GO environment variable.
for example:
export GO='${go_min_version}'
"
readonly go_min_version go_min_version_prefix go_version_msg
case "$( "$GO" version )"
in
("$go_min_version_prefix"*)
@ -74,15 +74,15 @@ esac
# Simple Analyzers
# blocklist_imports is a simple check against unwanted packages.
# Currently it only looks for package log which is replaced by our own
# package github.com/AdguardTeam/golibs/log.
# blocklist_imports is a simple check against unwanted packages. Currently it
# only looks for package log which is replaced by our own package
# github.com/AdguardTeam/golibs/log.
blocklist_imports() {
git grep -F -e '"log"' -- '*.go' || exit 0;
}
# method_const is a simple check against the usage of some raw strings
# and numbers where one should use named constants.
# method_const is a simple check against the usage of some raw strings and
# numbers where one should use named constants.
method_const() {
git grep -F -e '"GET"' -e '"POST"' -- '*.go' || exit 0;
}
@ -110,8 +110,8 @@ underscores() {
# Helpers
# exit_on_output exits with a nonzero exit code if there is anything in
# the command's combined output.
# exit_on_output exits with a nonzero exit code if there is anything in the
# command's combined output.
exit_on_output() (
set +e
@ -141,7 +141,7 @@ exit_on_output() (
echo "$output"
if [ "$exitcode" = '0' ]
if [ "$exitcode" -eq '0' ]
then
exitcode='1'
fi
@ -154,7 +154,8 @@ exit_on_output() (
# Constants
readonly go_files='./main.go ./tools.go ./internal/'
go_files='./main.go ./tools.go ./internal/'
readonly go_files
@ -181,8 +182,7 @@ ineffassign ./...
unparam ./...
git ls-files -- '*.go' '*.md' '*.mod' '*.sh' '*.yaml' '*.yml' 'Makefile'\
| xargs misspell --error
git ls-files -- '*.go' '*.mod' '*.sh' 'Makefile' | xargs misspell --error
looppointer ./...
@ -190,9 +190,8 @@ nilness ./...
exit_on_output shadow --strict ./...
# TODO(a.garipov): Enable errcheck fully after handling all errors,
# including the deferred and generated ones, properly. Also, perhaps,
# enable --blank.
# TODO(a.garipov): Enable errcheck fully after handling all errors, including
# the deferred and generated ones, properly. Also, perhaps, enable --blank.
#
# errcheck ./...
exit_on_output sh -c '

View File

@ -1,6 +1,7 @@
#!/bin/sh
verbose="${VERBOSE:-0}"
readonly verbose
# Verbosity levels:
# 0 = Don't print anything except for errors.
@ -21,22 +22,24 @@ else
v_flags=''
x_flags=''
fi
readonly v_flags x_flags
set -e -f -u
race="${RACE:-1}"
if [ "$race" = '0' ]
if [ "${RACE:-1}" -eq '0' ]
then
race_flags=''
else
race_flags='--race'
fi
readonly race_flags
readonly go="${GO:-go}"
readonly timeout_flags="${TIMEOUT_FLAGS:---timeout 30s}"
readonly cover_flags='--coverprofile ./coverage.txt'
readonly count_flags='--count 1'
go="${GO:-go}"
timeout_flags="${TIMEOUT_FLAGS:---timeout 30s}"
cover_flags='--coverprofile ./coverage.txt'
count_flags='--count 1'
readonly go timeout_flags cover_flags count_flags
# Don't use quotes with flag variables because we want an empty space if
# those aren't set.
# Don't use quotes with flag variables because we want an empty space if those
# aren't set.
"$go" test $count_flags $cover_flags $race_flags $timeout_flags $x_flags $v_flags ./...

View File

@ -1,33 +1,35 @@
#!/bin/sh
verbose="${VERBOSE:-0}"
readonly verbose
if [ "$verbose" -gt '1' ]
then
set -x
readonly v_flags='-v'
readonly x_flags='-x'
v_flags='-v'
x_flags='-x'
elif [ "$verbose" -gt '0' ]
then
set -x
readonly v_flags='-v'
readonly x_flags=''
v_flags='-v'
x_flags=''
else
set +x
readonly v_flags=''
readonly x_flags=''
v_flags=''
x_flags=''
fi
readonly v_flags x_flags
set -e -f -u
go="${GO:-go}"
readonly go
# TODO(a.garipov): Add goconst?
# Reset GOARCH and GOOS to make sure we install the tools for the native
# architecture even when we're cross-compiling the main binary, and also
# to prevent the "cannot install cross-compiled binaries when GOBIN is
# set" error.
# architecture even when we're cross-compiling the main binary, and also to
# prevent the "cannot install cross-compiled binaries when GOBIN is set" error.
env\
GOARCH=""\
GOOS=""\

49
scripts/make/txt-lint.sh Normal file
View File

@ -0,0 +1,49 @@
#!/bin/sh
verbose="${VERBOSE:-0}"
readonly verbose
# Set verbosity.
if [ "$verbose" -gt '0' ]
then
set -x
fi
# Set $EXIT_ON_ERROR to zero to see all errors.
if [ "${EXIT_ON_ERROR:-1}" -eq '0' ]
then
set +e
else
set -e
fi
# We don't need glob expansions and we want to see errors about unset variables.
set -f -u
# Deferred Helpers
not_found_msg='
looks like a binary not found error.
make sure you have installed the linter binaries using:
$ make go-tools
'
readonly not_found_msg
# TODO(a.garipov): Put it into a separate script and source it both here and in
# go-lint.sh?
not_found() {
if [ "$?" -eq '127' ]
then
# Code 127 is the exit status a shell uses when a command or
# a file is not found, according to the Bash Hackers wiki.
#
# See https://wiki.bash-hackers.org/dict/terms/exit_status.
echo "$not_found_msg" 1>&2
fi
}
trap not_found EXIT
git ls-files -- '*.md' '*.yaml' '*.yml' | xargs misspell --error

View File

@ -2,28 +2,29 @@
# AdGuard Home Version Generation Script
#
# This script generates versions based on the current git tree state.
# The valid output formats are:
# This script generates versions based on the current git tree state. The valid
# output formats are:
#
# * For release versions, "v0.123.4". This version should be the one
# in the current tag, and the script merely checks, that the current
# * For release versions, "v0.123.4". This version should be the one in the
# current tag, and the script merely checks, that the current commit is
# properly tagged.
#
# * For prerelease beta versions, "v0.123.4-b.5". This version should be the
# one in the current tag, and the script merely checks, that the current
# commit is properly tagged.
#
# * For prerelease beta versions, "v0.123.4-b.5". This version should
# be the one in the current tag, and the script merely checks, that
# the current commit is properly tagged.
# * For prerelease alpha versions (aka snapshots), "v0.123.4-a.6+a1b2c3d4".
#
# * For prerelease alpha versions (aka snapshots),
# "v0.123.4-a.6+a1b2c3d4".
# BUG(a.garipov): The script currently can't differentiate between beta tags and
# release tags if they are on the same commit, so the beta tag **must** be
# pushed and built **before** the release tag is pushed.
#
# BUG(a.garipov): The script currently can't differentiate between beta
# tags and release tags if they are on the same commit, so the beta tag
# **must** be pushed and built **before** the release tag is pushed.
#
# TODO(a.garipov): The script currently doesn't handle release branches,
# so it must be modified once we have those.
# TODO(a.garipov): The script currently doesn't handle release branches, so it
# must be modified once we have those.
verbose="${VERBOSE:-0}"
readonly verbose
readonly verbose="${VERBOSE:-0}"
if [ "$verbose" -gt '0' ]
then
set -x
@ -31,9 +32,9 @@ fi
set -e -f -u
# bump_minor is an awk program that reads a minor release version,
# increments the minor part of it, and prints the next version.
readonly bump_minor='/^v[0-9]+\.[0-9]+\.0$/ {
# bump_minor is an awk program that reads a minor release version, increments
# the minor part of it, and prints the next version.
bump_minor='/^v[0-9]+\.[0-9]+\.0$/ {
print($1 "." $2 + 1 ".0");
next;
@ -44,21 +45,19 @@ readonly bump_minor='/^v[0-9]+\.[0-9]+\.0$/ {
exit 1;
}'
readonly bump_minor
# get_last_minor_zero returns the last new minor release.
get_last_minor_zero() {
# List all tags. Then, select those that fit the pattern of
# a new minor release: a semver version with the patch part set
# to zero.
# List all tags. Then, select those that fit the pattern of a new minor
# release: a semver version with the patch part set to zero.
#
# Then, sort them first by the first field ("1"), starting with
# the second character to skip the "v" prefix (".2"), and only
# spanning the first field (",1"). The sort is numeric and
# reverse ("nr").
# Then, sort them first by the first field ("1"), starting with the
# second character to skip the "v" prefix (".2"), and only spanning the
# first field (",1"). The sort is numeric and reverse ("nr").
#
# Then, sort them by the second field ("2"), and only spanning
# the second field (",2"). The sort is also numeric and reverse
# ("nr").
# Then, sort them by the second field ("2"), and only spanning the
# second field (",2"). The sort is also numeric and reverse ("nr").
#
# Finally, get the top (that is, most recent) version.
git tag\
@ -67,7 +66,8 @@ get_last_minor_zero() {
| head -n 1
}
readonly channel="$CHANNEL"
channel="$CHANNEL"
readonly channel
case "$channel"
in
@ -77,32 +77,36 @@ in
;;
('edge')
# last_minor_zero is the last new minor release.
readonly last_minor_zero="$(get_last_minor_zero)"
last_minor_zero="$( get_last_minor_zero )"
readonly last_minor_zero
# num_commits_since_minor is the number of commits since the
# last new minor release. If the current commit is the new
# minor release, num_commits_since_minor is zero.
readonly num_commits_since_minor="$(git rev-list "${last_minor_zero}..HEAD" | wc -l)"
# num_commits_since_minor is the number of commits since the last new
# minor release. If the current commit is the new minor release,
# num_commits_since_minor is zero.
num_commits_since_minor="$( git rev-list "${last_minor_zero}..HEAD" | wc -l )"
readonly num_commits_since_minor
# next_minor is the next minor release version.
readonly next_minor="$(echo "$last_minor_zero" | awk -F '.' "$bump_minor")"
next_minor="$( echo "$last_minor_zero" | awk -F '.' "$bump_minor" )"
readonly next_minor
# Make this commit a prerelease version for the next minor
# release. For example, if the last minor release was v0.123.0,
# and the current commit is the fifth since then, the version
# will look something like:
# Make this commit a prerelease version for the next minor release. For
# example, if the last minor release was v0.123.0, and the current
# commit is the fifth since then, the version will look something like:
#
# v0.124.0-a.5+a1b2c3d4
#
version="${next_minor}-a.${num_commits_since_minor}+$(git rev-parse --short HEAD)"
version="${next_minor}-a.${num_commits_since_minor}+$( git rev-parse --short HEAD )"
;;
('beta'|'release')
# current_desc is the description of the current git commit. If
# the current commit is tagged, git describe will show the tag.
readonly current_desc="$(git describe)"
# current_desc is the description of the current git commit. If the
# current commit is tagged, git describe will show the tag.
current_desc="$( git describe )"
readonly current_desc
# last_tag is the most recent git tag.
readonly last_tag="$(git describe --abbrev=0)"
last_tag="$( git describe --abbrev=0 )"
readonly last_tag
# Require an actual tag for the beta and final releases.
if [ "$current_desc" != "$last_tag" ]

View File

@ -1,10 +1,12 @@
#!/bin/bash
# Get admin tool port from configuration
bind_port=$(grep bind_port $SNAP_DATA/AdGuardHome.yaml | awk -F ' ' '{print $2}')
#!/bin/sh
if [ -z "$bind_port" ]; then
xdg-open http://localhost:3000
# Get the admin interface port from the configuration.
bind_port="$( grep -e 'bind_port' "${SNAP_DATA}/AdGuardHome.yaml" | awk -F ' ' '{print $2}' )"
readonly bind_port
if [ "$bind_port" = '' ]
then
xdg-open 'http://localhost:3000'
else
xdg-open http://localhost:$bind_port
xdg-open "http://localhost:${bind_port}"
fi