playwright/browser_patches
Andrey Lushnikov 3486f1089f
devops: introduce independent linux build environments (#16470)
This patch introduces independent docker-based build environments for WebKit and Firefox
browsers.

To launch an interactive session inside the environment:

```sh
./browser_patches/docker/cli.sh webkit-debian-11 build # builds image
./browser_patches/docker/cli.sh webkit-debian-11 enter # launches interactive container
```
2022-08-13 04:40:45 -07:00
..
chromium browser(chromium): roll to 2022-Aug-11 (#16444) 2022-08-11 12:06:29 +02:00
chromium-tip-of-tree browser(chromium-tip-of-tree): roll to 2022-Aug-11 (#16443) 2022-08-11 12:06:39 +02:00
docker devops: introduce independent linux build environments (#16470) 2022-08-13 04:40:45 -07:00
ffmpeg chore: consolidate xcode versions across operating systems and builds (#13708) 2022-04-25 06:35:26 -07:00
firefox browser(firefox): cross thread sync in screencast (#16320) 2022-08-05 15:25:26 -07:00
firefox-beta browser(ff-beta): fix packaging manifest (#16451) 2022-08-11 05:13:27 -07:00
webkit chore: use existing is_linux function (#16509) 2022-08-12 12:38:19 -07:00
winldd chore: use helper functions to define platforms (#13707) 2022-04-22 12:35:35 -07:00
build.sh devops: fix //browser_patches/{build.sh,clean.sh} scripts (#9515) 2021-10-14 11:48:01 -07:00
checkout_build_archive_upload.sh devops: remove universal WebKit build (#16468) 2022-08-11 10:12:45 -07:00
clean.sh devops: fix //browser_patches/{build.sh,clean.sh} scripts (#9515) 2021-10-14 11:48:01 -07:00
export.sh devops: fix export.sh script (#14494) 2022-05-31 00:55:26 -07:00
get_xcode_version.js devops: fallback to xcode 13.2 for webkit (#14052) 2022-05-10 04:05:06 -07:00
prepare_checkout.sh devops: fix checkout re-use for multiple different base branches (#15231) 2022-06-29 08:30:39 -07:00
README.md browser(firefox): make scrollIntoViewIfNeeded and getContentQuads work with display:contents (#16111) 2022-08-01 14:40:23 -07:00
repack-juggler.mjs devops: adapt repack-juggler script to work with win (#15254) 2022-06-30 11:04:59 -07:00
sanitize_and_compress_log.js devops: use node.js to gzip logs 2020-04-20 02:52:26 -07:00
send_telegram_message.js devops: send Telegram messages via Node.js (#15804) 2022-07-20 15:45:26 -07:00
upload.sh chore: use helper functions to define platforms (#13707) 2022-04-22 12:35:35 -07:00
utils.sh devops: be compatible with multiple MSVS installations (#16121) 2022-08-01 23:37:07 +02:00

Contributing Browser Patches

Firefox and WebKit have additional patches atop to expose necessary capabilities.

Ideally, all these changes should be upstreamed. For the time being, it is possible to setup a browser checkout and develop from there.

WebKit upstream status

1. Setting up local browser checkout

From the playwright repo, run the following command:

$ ./browser_patches/prepare_checkout.sh firefox

(you can optionally pass "webkit" for a webkit checkout)

This will create a firefox checkout at $HOME/firefox

NOTE: this command downloads GBs of data.

This command will:

  • create a browser_upstream remote in the checkout
  • create a playwright-build branch and apply all playwright-required patches to it.

2. Developing a new change

Creating new branch

You want to create a new branch off the playwright-build branch.

Assuming that you're under $HOME/firefox checkout:

$ git checkout -b my-new-feature playwright-build
$ # develop my feature on the my-new-feature branch ....

Building

Each browser has corresponding build script. --full options normally takes care of also installing required build dependencies on Linux.

./browser_patches/firefox/build.sh --full

Running tests with local browser build

Playwright test suite may run against local browser build without bundling it.

# Run webkit tests with local webkit build
WKPATH=./browser_patches/webkit/pw_run.sh npm run wtest

# Run firefox tests with local firefox build on macos
FFPATH=/tmp/repackaged-firefox/firefox/Nightly.app/Contents/MacOS/firefox npm run ftest

# Run chromium tests with local chromium build on linux
CRPATH=~/chromium/src/out/Release/chrome npm run ctest

Flakiness dashboard

You can look at the flakiness dashboard to see recent history of any playwright test.

3. Exporting your change to playwright repo

Once you're happy with the work you did in the browser-land, you want to export it to the playwright repo.

Assuming that you're in the root of the playwright repo and that your browser checkout has your feature branch checked out:

$ ./browser_patches/export.sh firefox

This script will:

  • create a new patch and put it to the ./browser_patches/firefox/patches/
  • update the ./browser_patches/firefox/UPSTREAM_CONFIG.sh if necessary
  • bump the ./browser_patches/firefox/BUILD_NUMBER number.

The script will assume Firefox checkout is located at $HOME/firefox

Send a PR to the Playwright repo to be reviewed.

4. Rolling Playwright to the new browser build

Once the patch has been committed, the build bots will kick in, compile and upload a new browser version to all the platforms. Then you can roll the browser:

$ node utils/roll_browser.js chromium 123456

Cheatsheet

See browser stdout/stderr

Set the DEBUG=pw:browser environment variable to see it.

Firefox

Debug build

When compiling set the FF_DEBUG_BUILD=1 environment variable.

Stack trace

In //mozglue/misc/StackWalk.cpp add

#define MOZ_DEMANGLE_SYMBOLS 1

In native code use

#include "mozilla/StackWalk.h"
// ...
MozWalkTheStack(stderr);

If the stack trace is still mangled cat it to tools/rb/fix_linux_stack.py

Logging

Upstream documentation: https://firefox-source-docs.mozilla.org/xpcom/logging.html

MOZ_LOG=nsHttp:5

Module name is a string passed to the mozilla::LazyLogModule of the corresponding component, e.g.:

LazyLogModule gHttpLog("nsHttp");

Inside Juggler, you can use dump('foo\n').

WebKit

Logging

Inside Objective-C you can use NSLog.

NSLog(@"Foobar value: %@", value);

Debugging windows

In Source\WTF\wtf\win\DbgHelperWin.cpp replace

#if !defined(NDEBUG) with #if 1

Then regular WTFReportBacktrace() works.

Debugging linux

WTFReportBacktrace() has been broken since r283707, see this comment. Revert that change locally to make backtraces work again. Otherwise addr2line -f can still be used to map addresses to function names.

Enable core dumps on Linux

mkdir -p /tmp/coredumps
sudo bash -c 'echo "/tmp/coredumps/core-pid_%p.dump" > /proc/sys/kernel/core_pattern'
ulimit -c unlimited

Then to read stack traces run the following command:

# To find out crashing process name
file core-pid_29652.dump
# Point gdb to the local binary of the crashed process and the core file
gdb $HOME/.cache/ms-playwright/webkit-1292/minibrowser-gtk/WebKitWebProcess core-pid_29652
# Inside gdb update .so library search path to the local one
set solib-search-path /home/yurys/.cache/ms-playwright/webkit-1292/minibrowser-gtk
# Finally print backtrace
bt