1
1
mirror of https://github.com/mgree/ffs.git synced 2024-07-14 14:50:23 +03:00
ffs/tests/unpack_pack_symlink.sh
Dan Liu 3857d74d27
pack/unpack (#65)
* scoping out multi-binary stuff

* todo notes for pack/unpack [ci skip]

* new version of r in gh action

* created lib.rs and moved ffs main.rs bin. added bfs traversal of json structure.

* moved main.rs last commit. forgot to commit the removal

* added file creation for null,bool,num,str

* main.rs > ffs.rs to have ffs as executable

* error condition for already existing root dir and text files

* first cut at BFS with nodes

* unpack supports all 3 types now

* use config object for file name parsing, change from_args to from_ffs_args in ffs.rs

* fixed unpack configs

* added from_pack_args and some code to get the Value structure from a file

* added dep walkdir

* not working pack.rs

* remove dep walkdir

* almost working pack.rs

* pack works, but not too efficient

* change pack to return name and value so recursion can work without queue

* pack.rs similar to as_other_value now and works

* unpack and pack now have their own args parsers

* changed cli options for pack-unpack (unfinished)

* edited config to better support pack and unpack actions

* fixed unpacking into empty dir. only unpacking into unempty dir errors

* checked almost all error statuses

* fix no-xattr option for unpack

* prevent non-object/array from being unpacked

* fixed missing RUST_LOG warnings

* roundtrip tests for formats. since comments and formatting aren't preserved, unpack and pack twice

* fix pack: added --exact

* rm ERR_MSG tmp file in all tests. added all possible format conversion tests. edit to run_tests.sh to support new tests

* fix packunpack tests

* another update to test scripts

* unpack: don't remove dir if value is not map or list. pack: detect file type without xattr. started adapting more scripts to use unpack/pack

* changed user.type xattr for non-lists from 'map' to 'named'. converted 10 scripts

* added more scripts. fixed bug in infer_mount_relative fail()

* more tests, fix: original_name xattr only gets used if name is invalid

* more tests, missing 1 fail message in yaml_output test, changed pack list sorting to use file name instead of parsed integers to match ffs

* fixed tests for unpack/pack

* edit script formatting using quotes around vars, rm readonly for unpack

* pack: added back ignored file for lists and --no-xattr option in cli, added macos_noxattr_cleanup (not exactly the same as ffs test)

* Run `pack/`unpack` in macOS CI; factor out benchmarks (#63)

* added quiet inplace, umask test based on mode.sh

* added fail (un)pack for every call, fix missing n in fail msg for basic_object_exact

* fixed issues with adding fail conditions

* added (un)pack exit status tests

* added symlink support for pack

* symlinks mostly done. cleanup + efficiency checks needed

* added test for packing symlinks, added some comments for pack

* fix test4 for symlink test on linux

* see why test5 not working on linux

* actually print out the xattr

* fix: setting xattr on symlink doesn't work in linux :( making it macOS-specific

* impl --max-depth and --allow-symlink-escape. tests needed

* fix: macos links /var to /private/var so checking if symlinked path starts with mount errors. canonicalizing mount works.

* fix: wrong detected type map instead of named. add: symlink escape and maxdepth test.

* code cleanup, added test for symlink escape and maxdepth together, show warnings in config for pack/unpack not just errors

* simple changes for requests:
pack:122 add loop to error msg
debug! for received config in unpack & pack
f.write(s)? instead of write!
shadowed original_name
verb agreement in cli.rs
remove reserve for BTreeMap and now useless TODOs

* while let instead of queue.empty

* use auto instead of detect and check for auto and is_dir to resolve directory type
use .as_ref instead of .clone for accessing mount
remove non-symlinks from mapping

* resolve directory type always. warn for unknown path_type.

* warn when hitting broken symlinks.

* better warn message for broken symlink

* resolve repeated traversal of broken symlinks
store bool of whether link is broken in symlink mapping
checks symlinks a maximum of two times for broken links

* use struct instead of tuple in symlink map
for better naming and code clarity

* loosen criterion for determining directory type
directories get resolved as list if all files begin with an integer.
if a directory's user.type gets forcibly set to list without obeying
that property, all filenames that don't begin with an integer get put
to the end of the list, but are still sorted alphabetically.

* don't use regex to detect. just check first or first two chars manually.
continue to use regex for sorting because lexicographic sorting for files starting with -
means larger negative numbers go to the right

---------

Co-authored-by: Michael Greenberg <michael.greenberg@stevens.edu>
2023-09-27 14:02:15 +00:00

290 lines
6.3 KiB
Bash
Executable File

#!/bin/sh
fail() {
echo FAILED: $1
if [ "$MNT" ]
then
rm "$EXP" "$OUT"
rm -r "$MNT"
fi
exit 1
}
if [ "$RUNNER_OS" = "Linux" ] || [ "$(uname)" = "Linux" ]; then
which getfattr || fail getfattr
which setfattr || fail setfattr
getattr() {
attr=$1
shift
getfattr -h -n "$attr" --only-values "$@"
}
setattr() {
attr="$1"
val="$2"
shift 2
setfattr -h -n "$attr" -v "$val" "$@"
}
listattr() {
getfattr -h --match=- "$@"
}
rmattr() {
attr=$1
shift
setfattr -h -x "$attr" "$@"
}
elif [ "$RUNNER_OS" = "macOS" ] || [ "$(uname)" = "Darwin" ]; then
listattr() {
xattr -s -l "$@"
}
getattr() {
attr=$1
shift
xattr -s -p "$attr" "$@"
}
setattr() {
attr="$1"
val="$2"
shift 2
xattr -s -w "$attr" "$val" "$@"
}
rmattr() {
attr=$1
shift
xattr -s -d "$attr" "$@"
}
else
fail os
fi
typeof() {
getattr user.type "$@"
}
MNT=$(mktemp -d)
EXP=$(mktemp)
OUT=$(mktemp)
mv "$OUT" "$OUT".json
OUT="$OUT".json
# chain of symlinks and symlink to directory
# test0
# ├── a
# ├── b -> a
# ├── c -> b
# ├── d -> c
# ├── e -> d
# ├── tree
# │ ├── about
# │ └── root
# └── treecopy -> tree
cd "$MNT"
echo 'a' >a
ln -s a b
ln -s b c
ln -s c d
ln -s d e
mkdir tree
ln -s tree treecopy
cd tree
echo 'tree about' >about
echo 'tree root' >root
printf '{"a":"a","tree":{"about":"tree about","root":"tree root"}}' >"$EXP"
pack -o "$OUT" -- "$MNT" || fail pack1
diff "$EXP" "$OUT" || fail "test0 no-follow"
printf '{"a":"a","b":"a","c":"a","d":"a","e":"a","tree":{"about":"tree about","root":"tree root"},"treecopy":{"about":"tree about","root":"tree root"}}' >"$EXP"
pack -o "$OUT" -L -- "$MNT" || fail pack2
diff "$EXP" "$OUT" || fail "test0 follow"
rm -r "$MNT"
mkdir "$MNT"
# symlinks in list directories
# test1
# ├── ascending
# │ ├── 0 -> 1
# │ ├── 1 -> 2
# │ ├── 2 -> 3
# │ ├── 3 -> 4
# │ └── 4
# └── descending
# ├── 0
# ├── 1 -> 0
# ├── 2 -> 1
# ├── 3 -> 2
# └── 4 -> 3
cd "$MNT"
mkdir ascending descending
cd ascending
echo '4' >4
ln -s 4 3
ln -s 3 2
ln -s 2 1
ln -s 1 0
cd ../descending
echo '0' >0
ln -s 0 1
ln -s 1 2
ln -s 2 3
ln -s 3 4
printf '{"ascending":[4],"descending":[0]}' >"$EXP"
pack -o "$OUT" -- "$MNT" || fail pack3
diff "$EXP" "$OUT" || fail "test1 no-follow"
printf '{"ascending":[4,4,4,4,4],"descending":[0,0,0,0,0]}' >"$EXP"
pack -o "$OUT" -L -- "$MNT" || fail pack4
diff "$EXP" "$OUT" || fail "test1 follow"
rm -r "$MNT"
mkdir "$MNT"
# relative and absolute path symlinks to some path in mount
# test2
# └── path
# └── to
# ├── other
# │ └── file
# │ └── data
# └── some
# └── link
# ├── abs -> "$MNT"/path/to/other/file/data
# └── rel -> ../../other/file/data
cd "$MNT"
mkdir -p path/to/some/link path/to/other/file
touch path/to/other/file/data
cd path/to/some/link
ln -s ../../other/file/data rel
ln -s "$MNT"/path/to/other/file/data abs
printf '{"path":{"to":{"other":{"file":{"data":null}},"some":{"link":{}}}}}' >"$EXP"
pack -o "$OUT" -- "$MNT" || fail pack5
diff "$EXP" "$OUT" || fail "test2 no-follow"
printf '{"path":{"to":{"other":{"file":{"data":null}},"some":{"link":{"abs":null,"rel":null}}}}}' >"$EXP"
pack -o "$OUT" -L -- "$MNT" || fail pack6
diff "$EXP" "$OUT" || fail "test2 follow"
rm -r "$MNT"
mkdir "$MNT"
# symlink pointing to ancestor error
# test3
# └── path
# └── to
# ├── other
# │ └── file
# │ └── data
# └── some
# └── link
# └── linkfile -> ../../some
cd "$MNT"
mkdir -p path/to/some/link path/to/other/file
touch path/to/other/file/data
cd path/to/some/link
ln -s ../../some linkfile
printf '{"path":{"to":{"other":{"file":{"data":null}},"some":{"link":{}}}}}' >"$EXP"
pack -o "$OUT" -- "$MNT" || fail pack7
diff "$EXP" "$OUT" || fail "test3 no-follow"
pack -L -- "$MNT" >/dev/null 2>"$OUT" && fail "pack8 symlink to ancestor error"
cat "$OUT" | grep "ancestor directory" >/dev/null 2>&1 || fail "test3 follow expected error"
rm -r "$MNT"
mkdir "$MNT"
# symlink loop
# test4
# ├── a -> b
# ├── b -> a
# ├── c -> b
# ├── d -> c
# ├── e -> d
# └── f -> e
cd "$MNT"
ln -s a b
ln -s b c
ln -s c d
ln -s d e
ln -s e f
ln -s b a
printf '{}' >"$EXP"
pack -o "$OUT" -- "$MNT" || fail pack9
diff "$EXP" "$OUT" || fail "test4 no-follow"
pack -L -- "$MNT" >/dev/null 2>"$OUT" && fail "pack10 symlink loop error"
cat "$OUT" | grep "Symlink loop detected" >/dev/null 2>&1 || fail "test4 follow expected error"
if [ "$RUNNER_OS" = "macOS" ] || [ "$(uname)" = "Darwin" ]; then
rm -r "$MNT"
mkdir "$MNT"
# xattr propagates up the symlink chain unless redefined
# setting xattr for symlinks in linux doesn't work
# test5
# ├── a
# ├── b -> a
# ├── c -> b
# ├── d -> c
# ├── e -> d
# └── f -> e
cd "$MNT"
echo '4' >a
ln -s a b
ln -s b c
ln -s c d
ln -s d e
ln -s e f
setattr user.type integer a
setattr user.type string c
setattr user.type bytes e
printf '{"a":4}' >"$EXP"
pack -o "$OUT" -- "$MNT" || fail pack11
diff "$EXP" "$OUT" || fail "test5 no-follow"
printf '{"a":4,"b":4,"c":"4","d":"4","e":"NAo=","f":"NAo="}' >"$EXP"
pack -o "$OUT" -L -- "$MNT" || fail pack12
diff "$EXP" "$OUT" || fail "test5 follow"
fi
rm -r "$MNT"
mkdir "$MNT"
# test for allowing symlink to escape packed directory
# test6
# ├── a
# │ ├── a
# │ ├── b
# │ └── c -> ../b/c
# └── b
# └── c
cd "$MNT"
mkdir a b
echo "a" >a/a
echo "b" >a/b
echo "c" >b/c
cd a
ln -s ../b/c c
pack -L -- "$MNT"/a >/dev/null 2>"$OUT" || fail pack13
cat "$OUT" | grep "Specify --allow-symlink-escape" >/dev/null 2>&1 || fail "test6 follow but no escape"
printf '{"a":"a","b":"b","c":"c"}' >"$EXP"
pack -L --allow-symlink-escape -- "$MNT"/a >"$OUT" || fail pack14
diff "$EXP" "$OUT" || fail "test6 follow and escape"
rm "$EXP" "$OUT"
rm -r "$MNT"