ilyakooo0/nixpkgs
1
0
Fork 0
mirror of https://github.com/ilyakooo0/nixpkgs.git synced 2024-11-13 09:17:07 +03:00
Code Issues Projects Releases Wiki Activity
52fc21b10f
nixpkgs/nixos/tests/chromium.nix

293 lines
8.4 KiB
Nix
Raw Normal View History

nixos/tests/chromium: Allow overriding channel map This has been the case before e45c211, but it turns out that it's very useful to override the channel packages so we can run tests with different Chromium build options. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-21 00:20:39 +03:00
{ system ? builtins.currentSystem
tests: refactor to carry the package set as an argument This way, the package set will be possible to pass without re-importing all the time
2018-11-11 11:41:11 +03:00
, config ? {}
, pkgs ? import ../.. { inherit system config; }
nixos/tests/chromium: Add ungoogled-chromium and Google Chrome By default only `chromium` will be tested but other "channels" can be selected using e.g.: nix-build nixos/tests/chromium.nix -A ungoogled This also adds me as secondary maintainer (I'd like to get notified on PRs/issues and can review them).
2021-01-18 17:16:27 +03:00
, channelMap ? { # Maps "channels" to packages
stable = pkgs.chromium;
beta = pkgs.chromiumBeta;
dev = pkgs.chromiumDev;
ungoogled = pkgs.ungoogled-chromium;
chrome-stable = pkgs.google-chrome;
chrome-beta = pkgs.google-chrome-beta;
chrome-dev = pkgs.google-chrome-dev;
nixos/tests/chromium: Allow overriding channel map This has been the case before e45c211, but it turns out that it's very useful to override the channel packages so we can run tests with different Chromium build options. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-21 00:20:39 +03:00
}
}:
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 02:20:27 +03:00
nixosTests.chromium: Port to Python
2020-02-06 16:50:47 +03:00
with import ../lib/testing-python.nix { inherit system pkgs; };
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 02:20:27 +03:00
with pkgs.lib;
mapAttrs (channel: chromiumPkg: makeTest rec {
name = "chromium-${channel}";
chromium tests: inherit timeout from the package /cc #49442. It should decrease the waste of resources due to abortions.
2018-11-01 22:13:38 +03:00
meta = {
nixos/tests/chromium: Add ungoogled-chromium and Google Chrome By default only `chromium` will be tested but other "channels" can be selected using e.g.: nix-build nixos/tests/chromium.nix -A ungoogled This also adds me as secondary maintainer (I'd like to get notified on PRs/issues and can review them).
2021-01-18 17:16:27 +03:00
maintainers = with maintainers; [ aszlig primeos ];
chromium tests: inherit timeout from the package /cc #49442. It should decrease the waste of resources due to abortions.
2018-11-01 22:13:38 +03:00
# https://github.com/NixOS/hydra/issues/591#issuecomment-435125621
inherit (chromiumPkg.meta) timeout;
all tests: added meta.maintainers section
2015-07-12 13:09:40 +03:00
};
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 20:41:56 +03:00
nixos/tests/chromium: Detect popup using OCR. This will make the test a lot more reliable, because we no longer need to press ESC multiple times hoping that it will close the popup. Unfortunately in order to run this test I needed to locally revert the gyp update from a305e6855dd8723683c77635f45ae28411c8f36c. With the old gyp version however the test runs fine and it's able to properly detect the popup. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-05-22 12:23:09 +03:00
enableOCR = true;
nixosTests.chromium: Port to Python
2020-02-06 16:50:47 +03:00
user = "alice";
nixos/tests/chromium: Run tests as normal user The tests have failed because Chromium has started up displaying the following error message in a dialog window: Chromium can not be run as root. Please start Chromium as a normal user. If you need to run as root for development, rerun with the --no-sandbox flag. So let's run as user "alice" and pass all commands using the small helper function "ru" (to keep it short, it's for "Run as User"). Tested it by running the "stable" test on x86_64-linux. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Reported-by: @globin
2017-02-07 09:36:56 +03:00
machine.imports = [ ./common/user-account.nix ./common/x11.nix ];
Grmbl http://hydra.nixos.org/build/24983009
2015-08-25 12:26:32 +03:00
machine.virtualisation.memorySize = 2047;
nixosTests.chromium: Port to Python
2020-02-06 16:50:47 +03:00
machine.test-support.displayManager.auto.user = user;
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 02:20:27 +03:00
machine.environment.systemPackages = [ chromiumPkg ];
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 20:41:56 +03:00
startupHTML = pkgs.writeText "chromium-startup.html" ''
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Chromium startup notifier</title>
</head>
<body onload="javascript:document.title='startup done'">
<img src="file://${pkgs.fetchurl {
treewide: use https for nixos.org and hydra.nixos.org tarballs.nixos.org is omitted from the change because urls from there are always hashed and checked
2020-04-18 23:51:19 +03:00
url = "https://nixos.org/logo/nixos-hex.svg";
nixos/tests/chromium: Fix nixos-hex.svg hash
2020-01-07 21:42:52 +03:00
sha256 = "07ymq6nw8kc22m7kzxjxldhiq8gzmc7f45kq2bvhbdm0w5s112s4";
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 20:41:56 +03:00
}}" />
</body>
</html>
'';
testScript = let
xdo = name: text: let
xdoScript = pkgs.writeText "${name}.xdo" text;
in "${pkgs.xdotool}/bin/xdotool '${xdoScript}'";
in ''
nixosTests.chromium: Port to Python
2020-02-06 16:50:47 +03:00
import shlex
from contextlib import contextmanager, _GeneratorContextManager
nixos/tests/chromium: Run tests as normal user The tests have failed because Chromium has started up displaying the following error message in a dialog window: Chromium can not be run as root. Please start Chromium as a normal user. If you need to run as root for development, rerun with the --no-sandbox flag. So let's run as user "alice" and pass all commands using the small helper function "ru" (to keep it short, it's for "Run as User"). Tested it by running the "stable" test on x86_64-linux. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Reported-by: @globin
2017-02-07 09:36:56 +03:00
# Run as user alice
nixosTests.chromium: Port to Python
2020-02-06 16:50:47 +03:00
def ru(cmd):
return "su - ${user} -c " + shlex.quote(cmd)
nixos/tests/chromium: Add ungoogled-chromium and Google Chrome By default only `chromium` will be tested but other "channels" can be selected using e.g.: nix-build nixos/tests/chromium.nix -A ungoogled This also adds me as secondary maintainer (I'd like to get notified on PRs/issues and can review them).
2021-01-18 17:16:27 +03:00
def get_browser_binary():
"""Returns the name of the browser binary."""
pname = "${getName chromiumPkg.name}"
if pname.find("chromium") != -1:
return "chromium" # Same name for all channels and ungoogled-chromium
if pname == "google-chrome":
return "google-chrome-stable"
if pname == "google-chrome-dev":
return "google-chrome-unstable"
# For google-chrome-beta and as fallback:
return pname
nixosTests.chromium: Port to Python
2020-02-06 16:50:47 +03:00
def create_new_win():
with machine.nested("Creating a new Chromium window"):
nixos/tests/chromium: Fix another potential race condition The create_new_win() function could open multiple windows when used incorrectly. This change makes sure that a new window will only be created if the main window could be selected successfully. This also ignores the out return values as they're never used.
2021-01-23 14:06:09 +03:00
status, _ = machine.execute(
nixosTests.chromium: Port to Python
2020-02-06 16:50:47 +03:00
ru(
"${xdo "new-window" ''
search --onlyvisible --name "startup done"
windowfocus --sync
windowactivate --sync
''}"
)
)
nixos/tests/chromium: Fix another potential race condition The create_new_win() function could open multiple windows when used incorrectly. This change makes sure that a new window will only be created if the main window could be selected successfully. This also ignores the out return values as they're never used.
2021-01-23 14:06:09 +03:00
if status == 0:
machine.execute(
ru(
"${xdo "new-window" ''
key Ctrl+n
''}"
)
nixosTests.chromium: Port to Python
2020-02-06 16:50:47 +03:00
)
def close_win():
def try_close(_):
nixos/tests/chromium: Fix a race condition Only execute Ctrl+w to close the currently active window if the new/secondary window (title: "New Tab") could be selected. This fixes a test failure since the update to Chromium M88 (cc PR #110010). Without this additional check the main window (title: "startup done") could still be selected (and thus will be closed) and the script would close both windows (i.e. terminate Chromium completely).
2021-01-20 18:42:40 +03:00
status, _ = machine.execute(
nixosTests.chromium: Port to Python
2020-02-06 16:50:47 +03:00
ru(
"${xdo "close-window" ''
search --onlyvisible --name "new tab"
windowfocus --sync
windowactivate --sync
''}"
)
)
nixos/tests/chromium: Fix a race condition Only execute Ctrl+w to close the currently active window if the new/secondary window (title: "New Tab") could be selected. This fixes a test failure since the update to Chromium M88 (cc PR #110010). Without this additional check the main window (title: "startup done") could still be selected (and thus will be closed) and the script would close both windows (i.e. terminate Chromium completely).
2021-01-20 18:42:40 +03:00
if status == 0:
machine.execute(
ru(
"${xdo "close-window" ''
key Ctrl+w
''}"
)
nixosTests.chromium: Port to Python
2020-02-06 16:50:47 +03:00
)
for _ in range(1, 20):
nixos/tests/chromium: Fix another potential race condition The create_new_win() function could open multiple windows when used incorrectly. This change makes sure that a new window will only be created if the main window could be selected successfully. This also ignores the out return values as they're never used.
2021-01-23 14:06:09 +03:00
status, _ = machine.execute(
nixosTests.chromium: Port to Python
2020-02-06 16:50:47 +03:00
ru(
"${xdo "wait-for-close" ''
search --onlyvisible --name "new tab"
''}"
)
)
if status != 0:
return True
machine.sleep(1)
return False
retry(try_close)
def wait_for_new_win():
ret = False
with machine.nested("Waiting for new Chromium window to appear"):
for _ in range(1, 20):
nixos/tests/chromium: Fix another potential race condition The create_new_win() function could open multiple windows when used incorrectly. This change makes sure that a new window will only be created if the main window could be selected successfully. This also ignores the out return values as they're never used.
2021-01-23 14:06:09 +03:00
status, _ = machine.execute(
nixosTests.chromium: Port to Python
2020-02-06 16:50:47 +03:00
ru(
"${xdo "wait-for-window" ''
search --onlyvisible --name "new tab"
windowfocus --sync
windowactivate --sync
''}"
)
)
if status == 0:
ret = True
machine.sleep(10)
break
machine.sleep(1)
return ret
def create_and_wait_for_new_win():
for _ in range(1, 3):
create_new_win()
if wait_for_new_win():
return True
assert False, "new window did not appear within 60 seconds"
@contextmanager
def test_new_win(description):
create_and_wait_for_new_win()
with machine.nested(description):
yield
close_win()
machine.wait_for_x()
url = "file://${startupHTML}"
nixos/tests/chromium: Add ungoogled-chromium and Google Chrome By default only `chromium` will be tested but other "channels" can be selected using e.g.: nix-build nixos/tests/chromium.nix -A ungoogled This also adds me as secondary maintainer (I'd like to get notified on PRs/issues and can review them).
2021-01-18 17:16:27 +03:00
machine.succeed(ru(f'ulimit -c unlimited; "{get_browser_binary()}" "{url}" & disown'))
if get_browser_binary().startswith("google-chrome"):
# Need to click away the first window:
machine.wait_for_text("Make Google Chrome the default browser")
machine.screenshot("google_chrome_default_browser_prompt")
machine.send_key("ret")
nixosTests.chromium: Port to Python
2020-02-06 16:50:47 +03:00
machine.wait_for_text("startup done")
machine.wait_until_succeeds(
ru(
"${xdo "check-startup" ''
search --sync --onlyvisible --name "startup done"
# close first start help popup
key -delay 1000 Escape
windowfocus --sync
windowactivate --sync
''}"
)
)
create_and_wait_for_new_win()
machine.screenshot("empty_windows")
close_win()
machine.screenshot("startup_done")
with test_new_win("check sandbox"):
machine.succeed(
ru(
"${xdo "type-url" ''
search --sync --onlyvisible --name "new tab"
windowfocus --sync
type --delay 1000 "chrome://sandbox"
''}"
)
)
machine.succeed(
ru(
"${xdo "submit-url" ''
search --sync --onlyvisible --name "new tab"
windowfocus --sync
key --delay 1000 Return
''}"
)
)
machine.screenshot("sandbox_info")
machine.succeed(
ru(
"${xdo "find-window" ''
search --sync --onlyvisible --name "sandbox status"
windowfocus --sync
''}"
)
)
machine.succeed(
ru(
"${xdo "copy-sandbox-info" ''
key --delay 1000 Ctrl+a Ctrl+c
''}"
)
)
clipboard = machine.succeed(
ru("${pkgs.xclip}/bin/xclip -o")
)
filters = [
"layer 1 sandbox.*namespace",
"pid namespaces.*yes",
"network namespaces.*yes",
"seccomp.*sandbox.*yes",
"you are adequately sandboxed",
]
if not all(
re.search(filter, clipboard, flags=re.DOTALL | re.IGNORECASE)
for filter in filters
):
assert False, f"sandbox not working properly: {clipboard}"
machine.sleep(1)
machine.succeed(
ru(
"${xdo "find-window-after-copy" ''
search --onlyvisible --name "sandbox status"
''}"
)
)
clipboard = machine.succeed(
ru(
"echo void | ${pkgs.xclip}/bin/xclip -i"
)
)
machine.succeed(
ru(
"${xdo "copy-sandbox-info" ''
key --delay 1000 Ctrl+a Ctrl+c
''}"
)
)
clipboard = machine.succeed(
ru("${pkgs.xclip}/bin/xclip -o")
)
if not all(
re.search(filter, clipboard, flags=re.DOTALL | re.IGNORECASE)
for filter in filters
):
assert False, f"copying twice in a row does not work properly: {clipboard}"
machine.screenshot("after_copy_from_chromium")
machine.shutdown()
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 20:41:56 +03:00
'';
nixos/tests/chromium: Allow overriding channel map This has been the case before e45c211, but it turns out that it's very useful to override the channel packages so we can run tests with different Chromium build options. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-21 00:20:39 +03:00
}) channelMap
Reference in New Issue Copy Permalink