mirror of
https://github.com/microsoft/playwright.git
synced 2024-11-28 09:23:42 +03:00
feat(reporters): align and document environment variables (#30912)
- Documents `PLAYWRIGHT_FORCE_TTY` and `FORCE_COLOR` across terminal reporters. - New `PLAYWRIGHT_LIST_PRINT_STEPS`. Removes undocumented test-only `PW_TEST_DEBUG_REPORTERS_PRINT_STEPS`. - Replaces `PLAYWRIGHT_HTML_REPORT` with `PLAYWRIGHT_HTML_OUTPUT_DIR` and `PW_TEST_HTML_REPORT_OPEN` with `PLAYWRIGHT_HTML_OPEN` for consistency, supports older versions for backwards compatibility. - New `PLAYWRIGHT_HTML_HOST`, `PLAYWRIGHT_HTML_PORT` and `PLAYWRIGHT_HTML_ATTACHMENTS_BASE_URL`. - New `PLAYWRIGHT_JUNIT_STRIP_ANSI` and `PLAYWRIGHT_JUNIT_INCLUDE_PROJECT_IN_TEST_NAME`. - Removes `PW_HTML_REPORT` that was set for unknown reason.
This commit is contained in:
Dmitry Gozman
GitHub
parent
a93ad3dade
commit
6290af3a08
@ -97,6 +97,15 @@ export default defineConfig({
|
||||
});
|
||||
```
|
||||
|
||||
List report supports the following configuration options and environment variables:
|
||||
|
||||
| Environment Variable Name | Reporter Config Option| Description | Default
|
||||
|---|---|---|---|
|
||||
| `PLAYWRIGHT_LIST_PRINT_STEPS` | `printSteps` | Whether to print each step on its own line. | `false`
|
||||
| `PLAYWRIGHT_FORCE_TTY` | | Whether to produce output suitable for a live terminal. | `true` when terminal is in TTY mode, `false` otherwise.
|
||||
| `FORCE_COLOR` | | Whether to produce colored output. | `true` when terminal is in TTY mode, `false` otherwise.
|
||||
|
||||
|
||||
### Line reporter
|
||||
|
||||
Line reporter is more concise than the list reporter. It uses a single line to report last finished test, and prints failures when they occur. Line reporter is useful for large test suites where it shows the progress but does not spam the output by listing all the tests.
|
||||
@ -127,6 +136,14 @@ Running 124 tests using 6 workers
|
||||
[23/124] gitignore.spec.ts - should respect nested .gitignore
|
||||
```
|
||||
|
||||
Line report supports the following configuration options and environment variables:
|
||||
|
||||
| Environment Variable Name | Reporter Config Option| Description | Default
|
||||
|---|---|---|---|
|
||||
| `PLAYWRIGHT_FORCE_TTY` | | Whether to produce output suitable for a live terminal. | `true` when terminal is in TTY mode, `false` otherwise.
|
||||
| `FORCE_COLOR` | | Whether to produce colored output. | `true` when terminal is in TTY mode, `false` otherwise.
|
||||
|
||||
|
||||
### Dot reporter
|
||||
|
||||
Dot reporter is very concise - it only produces a single character per successful test run. It is the default on CI and useful where you don't want a lot of output.
|
||||
@ -150,6 +167,15 @@ Running 124 tests using 6 workers
|
||||
······F·············································
|
||||
```
|
||||
|
||||
|
||||
Dot report supports the following configuration options and environment variables:
|
||||
|
||||
| Environment Variable Name | Reporter Config Option| Description | Default
|
||||
|---|---|---|---|
|
||||
| `PLAYWRIGHT_FORCE_TTY` | | Whether to produce output suitable for a live terminal. | `true` when terminal is in TTY mode, `false` otherwise.
|
||||
| `FORCE_COLOR` | | Whether to produce colored output. | `true` when terminal is in TTY mode, `false` otherwise.
|
||||
|
||||
|
||||
### HTML reporter
|
||||
|
||||
HTML reporter produces a self-contained folder that contains report for the test run that can be served as a web page.
|
||||
@ -159,7 +185,7 @@ npx playwright test --reporter=html
|
||||
```
|
||||
|
||||
By default, HTML report is opened automatically if some of the tests failed. You can control this behavior via the
|
||||
`open` property in the Playwright config or the `PW_TEST_HTML_REPORT_OPEN` environmental variable. The possible values for that property are `always`, `never` and `on-failure`
|
||||
`open` property in the Playwright config or the `PLAYWRIGHT_HTML_OPEN` environmental variable. The possible values for that property are `always`, `never` and `on-failure`
|
||||
(default).
|
||||
|
||||
You can also configure `host` and `port` that are used to serve the HTML report.
|
||||
@ -173,7 +199,7 @@ export default defineConfig({
|
||||
```
|
||||
|
||||
By default, report is written into the `playwright-report` folder in the current working directory. One can override
|
||||
that location using the `PLAYWRIGHT_HTML_REPORT` environment variable or a reporter configuration.
|
||||
that location using the `PLAYWRIGHT_HTML_OUTPUT_DIR` environment variable or a reporter configuration.
|
||||
|
||||
In configuration file, pass options directly:
|
||||
|
||||
@ -207,6 +233,16 @@ Or if there is a custom folder name:
|
||||
npx playwright show-report my-report
|
||||
```
|
||||
|
||||
HTML report supports the following configuration options and environment variables:
|
||||
|
||||
| Environment Variable Name | Reporter Config Option| Description | Default
|
||||
|---|---|---|---|
|
||||
| `PLAYWRIGHT_HTML_OUTPUT_DIR` | `outputFolder` | Directory to save the report to. | `playwright-report`
|
||||
| `PLAYWRIGHT_HTML_OPEN` | `open` | When to open the html report in the browser, one of `'always'`, `'never'` or `'on-failure'` | `'on-failure'`
|
||||
| `PLAYWRIGHT_HTML_HOST` | `host` | When report opens in the browser, it will be served bound to this hostname. | `localhost`
|
||||
| `PLAYWRIGHT_HTML_PORT` | `port` | When report opens in the browser, it will be served on this port. | `9323` or any available port when `9323` is not available.
|
||||
| `PLAYWRIGHT_HTML_ATTACHMENTS_BASE_URL` | `attachmentsBaseURL` | A separate location where attachments from the `data` subdirectory are uploaded. Only needed when you upload report and `data` separately to different locations. | `data/`
|
||||
|
||||
### Blob reporter
|
||||
|
||||
Blob reports contain all the details about the test run and can be used later to produce any other report. Their primary function is to facilitate the merging of reports from [sharded tests](./test-sharding.md).
|
||||
@ -308,8 +344,8 @@ JUnit report supports following configuration options and environment variables:
|
||||
| `PLAYWRIGHT_JUNIT_OUTPUT_DIR` | | Directory to save the output file. Ignored if output file is not specified. | `cwd` or config directory.
|
||||
| `PLAYWRIGHT_JUNIT_OUTPUT_NAME` | `outputFile` | Base file name for the output, relative to the output dir. | JUnit report is printed to the stdout.
|
||||
| `PLAYWRIGHT_JUNIT_OUTPUT_FILE` | `outputFile` | Full path to the output file. If defined, `PLAYWRIGHT_JUNIT_OUTPUT_DIR` and `PLAYWRIGHT_JUNIT_OUTPUT_NAME` will be ignored. | JUnit report is printed to the stdout.
|
||||
| | `stripANSIControlSequences` | Whether to remove ANSI control sequences from the text before writing it in the report. | By default output text is added as is.
|
||||
| | `includeProjectInTestName` | Whether to include Playwright project name in every test case as a name prefix. | By default not included.
|
||||
| `PLAYWRIGHT_JUNIT_STRIP_ANSI` | `stripANSIControlSequences` | Whether to remove ANSI control sequences from the text before writing it in the report. | By default output text is added as is.
|
||||
| `PLAYWRIGHT_JUNIT_INCLUDE_PROJECT_IN_TEST_NAME` | `includeProjectInTestName` | Whether to include Playwright project name in every test case as a name prefix. | By default not included.
|
||||
| `PLAYWRIGHT_JUNIT_SUITE_ID` | | Value of the `id` attribute on the root `<testsuites/>` report entry. | Empty string.
|
||||
| `PLAYWRIGHT_JUNIT_SUITE_NAME` | | Value of the `name` attribute on the root `<testsuites/>` report entry. | Empty string.
|
||||
|
||||
|
||||
@ -21,9 +21,13 @@ export function getFromENV(name: string): string | undefined {
|
||||
return value;
|
||||
}
|
||||
|
||||
export function getAsBooleanFromENV(name: string): boolean {
|
||||
export function getAsBooleanFromENV(name: string, defaultValue?: boolean | undefined): boolean {
|
||||
const value = getFromENV(name);
|
||||
return !!value && value !== 'false' && value !== '0';
|
||||
if (value === 'false' || value === '0')
|
||||
return false;
|
||||
if (value)
|
||||
return true;
|
||||
return !!defaultValue;
|
||||
}
|
||||
|
||||
export function getPackageManager() {
|
||||
|
||||
@ -62,14 +62,14 @@ class HtmlReporter extends EmptyReporter {
|
||||
private _outputFolder!: string;
|
||||
private _attachmentsBaseURL!: string;
|
||||
private _open: string | undefined;
|
||||
private _port: number | undefined;
|
||||
private _host: string | undefined;
|
||||
private _buildResult: { ok: boolean, singleTestId: string | undefined } | undefined;
|
||||
private _topLevelErrors: TestError[] = [];
|
||||
|
||||
constructor(options: HtmlReporterOptions) {
|
||||
super();
|
||||
this._options = options;
|
||||
if (options._mode === 'test')
|
||||
process.env.PW_HTML_REPORT = '1';
|
||||
}
|
||||
|
||||
override printsToStdio() {
|
||||
@ -81,9 +81,11 @@ class HtmlReporter extends EmptyReporter {
|
||||
}
|
||||
|
||||
override onBegin(suite: Suite) {
|
||||
const { outputFolder, open, attachmentsBaseURL } = this._resolveOptions();
|
||||
const { outputFolder, open, attachmentsBaseURL, host, port } = this._resolveOptions();
|
||||
this._outputFolder = outputFolder;
|
||||
this._open = open;
|
||||
this._host = host;
|
||||
this._port = port;
|
||||
this._attachmentsBaseURL = attachmentsBaseURL;
|
||||
const reportedWarnings = new Set<string>();
|
||||
for (const project of this.config.projects) {
|
||||
@ -104,12 +106,14 @@ class HtmlReporter extends EmptyReporter {
|
||||
this.suite = suite;
|
||||
}
|
||||
|
||||
_resolveOptions(): { outputFolder: string, open: HtmlReportOpenOption, attachmentsBaseURL: string } {
|
||||
_resolveOptions(): { outputFolder: string, open: HtmlReportOpenOption, attachmentsBaseURL: string, host: string | undefined, port: number | undefined } {
|
||||
const outputFolder = reportFolderFromEnv() ?? resolveReporterOutputPath('playwright-report', this._options.configDir, this._options.outputFolder);
|
||||
return {
|
||||
outputFolder,
|
||||
open: getHtmlReportOptionProcessEnv() || this._options.open || 'on-failure',
|
||||
attachmentsBaseURL: this._options.attachmentsBaseURL || 'data/'
|
||||
attachmentsBaseURL: process.env.PLAYWRIGHT_HTML_ATTACHMENTS_BASE_URL || this._options.attachmentsBaseURL || 'data/',
|
||||
host: process.env.PLAYWRIGHT_HTML_HOST || this._options.host,
|
||||
port: process.env.PLAYWRIGHT_HTML_PORT ? +process.env.PLAYWRIGHT_HTML_PORT : this._options.port,
|
||||
};
|
||||
}
|
||||
|
||||
@ -135,12 +139,12 @@ class HtmlReporter extends EmptyReporter {
|
||||
const { ok, singleTestId } = this._buildResult;
|
||||
const shouldOpen = !this._options._isTestServer && (this._open === 'always' || (!ok && this._open === 'on-failure'));
|
||||
if (shouldOpen) {
|
||||
await showHTMLReport(this._outputFolder, this._options.host, this._options.port, singleTestId);
|
||||
await showHTMLReport(this._outputFolder, this._host, this._port, singleTestId);
|
||||
} else if (this._options._mode === 'test') {
|
||||
const packageManagerCommand = getPackageManagerExecCommand();
|
||||
const relativeReportPath = this._outputFolder === standaloneDefaultFolder() ? '' : ' ' + path.relative(process.cwd(), this._outputFolder);
|
||||
const hostArg = this._options.host ? ` --host ${this._options.host}` : '';
|
||||
const portArg = this._options.port ? ` --port ${this._options.port}` : '';
|
||||
const hostArg = this._host ? ` --host ${this._host}` : '';
|
||||
const portArg = this._port ? ` --port ${this._port}` : '';
|
||||
console.log('');
|
||||
console.log('To open last HTML report run:');
|
||||
console.log(colors.cyan(`
|
||||
@ -151,18 +155,18 @@ class HtmlReporter extends EmptyReporter {
|
||||
}
|
||||
|
||||
function reportFolderFromEnv(): string | undefined {
|
||||
if (process.env[`PLAYWRIGHT_HTML_REPORT`])
|
||||
return path.resolve(process.cwd(), process.env[`PLAYWRIGHT_HTML_REPORT`]);
|
||||
return undefined;
|
||||
// Note: PLAYWRIGHT_HTML_REPORT is for backwards compatibility.
|
||||
const envValue = process.env.PLAYWRIGHT_HTML_OUTPUT_DIR || process.env.PLAYWRIGHT_HTML_REPORT;
|
||||
return envValue ? path.resolve(envValue) : undefined;
|
||||
}
|
||||
|
||||
function getHtmlReportOptionProcessEnv(): HtmlReportOpenOption | undefined {
|
||||
const processKey = 'PW_TEST_HTML_REPORT_OPEN';
|
||||
const htmlOpenEnv = process.env[processKey];
|
||||
// Note: PW_TEST_HTML_REPORT_OPEN is for backwards compatibility.
|
||||
const htmlOpenEnv = process.env.PLAYWRIGHT_HTML_OPEN || process.env.PW_TEST_HTML_REPORT_OPEN;
|
||||
if (!htmlOpenEnv)
|
||||
return undefined;
|
||||
if (!isHtmlReportOption(htmlOpenEnv)) {
|
||||
console.log(colors.red(`Configuration Error: HTML reporter Invalid value for ${processKey}: ${htmlOpenEnv}. Valid values are: ${htmlReportOptions.join(', ')}`));
|
||||
console.log(colors.red(`Configuration Error: HTML reporter Invalid value for PLAYWRIGHT_HTML_OPEN: ${htmlOpenEnv}. Valid values are: ${htmlReportOptions.join(', ')}`));
|
||||
return undefined;
|
||||
}
|
||||
return htmlOpenEnv;
|
||||
|
||||
@ -19,6 +19,7 @@ import path from 'path';
|
||||
import type { FullConfig, FullResult, Suite, TestCase } from '../../types/testReporter';
|
||||
import { formatFailure, resolveOutputFile, stripAnsiEscapes } from './base';
|
||||
import EmptyReporter from './empty';
|
||||
import { getAsBooleanFromENV } from 'playwright-core/lib/utils';
|
||||
|
||||
type JUnitOptions = {
|
||||
outputFile?: string,
|
||||
@ -42,8 +43,8 @@ class JUnitReporter extends EmptyReporter {
|
||||
|
||||
constructor(options: JUnitOptions) {
|
||||
super();
|
||||
this.stripANSIControlSequences = options.stripANSIControlSequences || false;
|
||||
this.includeProjectInTestName = options.includeProjectInTestName || false;
|
||||
this.stripANSIControlSequences = getAsBooleanFromENV('PLAYWRIGHT_JUNIT_STRIP_ANSI', !!options.stripANSIControlSequences);
|
||||
this.includeProjectInTestName = getAsBooleanFromENV('PLAYWRIGHT_JUNIT_INCLUDE_PROJECT_IN_TEST_NAME', !!options.includeProjectInTestName);
|
||||
this.configDir = options.configDir;
|
||||
this.resolvedOutputFile = resolveOutputFile('JUNIT', options)?.outputFile;
|
||||
}
|
||||
|
||||
@ -17,6 +17,7 @@
|
||||
import { ms as milliseconds } from 'playwright-core/lib/utilsBundle';
|
||||
import { colors, BaseReporter, formatError, formatTestTitle, isTTY, stepSuffix, stripAnsiEscapes, ttyWidth } from './base';
|
||||
import type { FullResult, Suite, TestCase, TestError, TestResult, TestStep } from '../../types/testReporter';
|
||||
import { getAsBooleanFromENV } from 'playwright-core/lib/utils';
|
||||
|
||||
// Allow it in the Visual Studio Code Terminal and the new Windows Terminal
|
||||
const DOES_NOT_SUPPORT_UTF8_IN_TERMINAL = process.platform === 'win32' && process.env.TERM_PROGRAM !== 'vscode' && !process.env.WT_SESSION;
|
||||
@ -33,9 +34,9 @@ class ListReporter extends BaseReporter {
|
||||
private _needNewLine = false;
|
||||
private _printSteps: boolean;
|
||||
|
||||
constructor(options: { omitFailures?: boolean, printSteps?: boolean } = {}) {
|
||||
super(options);
|
||||
this._printSteps = isTTY && (options.printSteps || !!process.env.PW_TEST_DEBUG_REPORTERS_PRINT_STEPS);
|
||||
constructor(options: { printSteps?: boolean } = {}) {
|
||||
super();
|
||||
this._printSteps = isTTY && getAsBooleanFromENV('PLAYWRIGHT_LIST_PRINT_STEPS', options.printSteps);
|
||||
}
|
||||
|
||||
override printsToStdio() {
|
||||
|
||||
@ -224,6 +224,7 @@ export function cleanEnv(env: NodeJS.ProcessEnv): NodeJS.ProcessEnv {
|
||||
GITHUB_SHA: undefined,
|
||||
// END: Reserved CI
|
||||
PW_TEST_HTML_REPORT_OPEN: undefined,
|
||||
PLAYWRIGHT_HTML_OPEN: undefined,
|
||||
PW_TEST_REPORTER: undefined,
|
||||
PW_TEST_REPORTER_WS_ENDPOINT: undefined,
|
||||
PW_TEST_SOURCE_TRANSFORM: undefined,
|
||||
|
||||
@ -209,7 +209,7 @@ test('should merge into html with dependencies', async ({ runInlineTest, mergeRe
|
||||
const reportFiles = await fs.promises.readdir(reportDir);
|
||||
reportFiles.sort();
|
||||
expect(reportFiles).toEqual([expect.stringMatching(/report-.*.zip/), expect.stringMatching(/report-.*.zip/), expect.stringMatching(/report-.*.zip/)]);
|
||||
const { exitCode, output } = await mergeReports(reportDir, { 'PW_TEST_HTML_REPORT_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
const { exitCode, output } = await mergeReports(reportDir, { 'PLAYWRIGHT_HTML_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
expect(exitCode).toBe(0);
|
||||
|
||||
expect(output).not.toContain('To open last HTML report run:');
|
||||
@ -280,7 +280,7 @@ test('should merge blob into blob', async ({ runInlineTest, mergeReports, showRe
|
||||
}
|
||||
{
|
||||
const compinedBlobReportDir = test.info().outputPath('blob-report');
|
||||
const { exitCode } = await mergeReports(compinedBlobReportDir, { 'PW_TEST_HTML_REPORT_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html,json'] });
|
||||
const { exitCode } = await mergeReports(compinedBlobReportDir, { 'PLAYWRIGHT_HTML_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html,json'] });
|
||||
expect(exitCode).toBe(0);
|
||||
expect(fs.existsSync(test.info().outputPath('report.json'))).toBe(true);
|
||||
await showReport();
|
||||
@ -335,7 +335,7 @@ test('be able to merge incomplete shards', async ({ runInlineTest, mergeReports,
|
||||
const reportFiles = await fs.promises.readdir(reportDir);
|
||||
reportFiles.sort();
|
||||
expect(reportFiles).toEqual([expect.stringMatching(/report-.*.zip/), expect.stringMatching(/report-.*.zip/)]);
|
||||
const { exitCode } = await mergeReports(reportDir, { 'PW_TEST_HTML_REPORT_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
const { exitCode } = await mergeReports(reportDir, { 'PLAYWRIGHT_HTML_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
expect(exitCode).toBe(0);
|
||||
|
||||
await showReport();
|
||||
@ -374,7 +374,7 @@ test('total time is from test run not from merge', async ({ runInlineTest, merge
|
||||
await runInlineTest(files, { shard: `1/2` });
|
||||
await runInlineTest(files, { shard: `2/2` }, { PWTEST_BLOB_DO_NOT_REMOVE: '1' });
|
||||
|
||||
const { exitCode, output } = await mergeReports(reportDir, { 'PW_TEST_HTML_REPORT_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
const { exitCode, output } = await mergeReports(reportDir, { 'PLAYWRIGHT_HTML_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
expect(exitCode).toBe(0);
|
||||
|
||||
expect(output).not.toContain('To open last HTML report run:');
|
||||
@ -441,7 +441,7 @@ test('merge into list report by default', async ({ runInlineTest, mergeReports }
|
||||
const reportFiles = await fs.promises.readdir(reportDir);
|
||||
reportFiles.sort();
|
||||
expect(reportFiles).toEqual(['report-1.zip', 'report-2.zip', 'report-3.zip']);
|
||||
const { exitCode, output } = await mergeReports(reportDir, { PW_TEST_DEBUG_REPORTERS: '1', PW_TEST_DEBUG_REPORTERS_PRINT_STEPS: '1', PLAYWRIGHT_FORCE_TTY: '80' }, { additionalArgs: ['--reporter', 'list'] });
|
||||
const { exitCode, output } = await mergeReports(reportDir, { PW_TEST_DEBUG_REPORTERS: '1', PLAYWRIGHT_LIST_PRINT_STEPS: '1', PLAYWRIGHT_FORCE_TTY: '80' }, { additionalArgs: ['--reporter', 'list'] });
|
||||
expect(exitCode).toBe(0);
|
||||
|
||||
const text = stripAnsi(output);
|
||||
@ -520,7 +520,7 @@ test('should print progress', async ({ runInlineTest, mergeReports }) => {
|
||||
const reportFiles = await fs.promises.readdir(reportDir);
|
||||
reportFiles.sort();
|
||||
expect(reportFiles).toEqual(['report-1.zip', 'report-2.zip']);
|
||||
const { exitCode, output } = await mergeReports(reportDir, { PW_TEST_HTML_REPORT_OPEN: 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
const { exitCode, output } = await mergeReports(reportDir, { PLAYWRIGHT_HTML_OPEN: 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
expect(exitCode).toBe(0);
|
||||
|
||||
const lines = output.split('\n');
|
||||
@ -571,7 +571,7 @@ test('preserve attachments', async ({ runInlineTest, mergeReports, showReport, p
|
||||
const reportFiles = await fs.promises.readdir(reportDir);
|
||||
reportFiles.sort();
|
||||
expect(reportFiles).toEqual(['report-1.zip']);
|
||||
const { exitCode } = await mergeReports(reportDir, { 'PW_TEST_HTML_REPORT_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
const { exitCode } = await mergeReports(reportDir, { 'PLAYWRIGHT_HTML_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
expect(exitCode).toBe(0);
|
||||
|
||||
await showReport();
|
||||
@ -634,7 +634,7 @@ test('generate html with attachment urls', async ({ runInlineTest, mergeReports,
|
||||
const reportFiles = await fs.promises.readdir(reportDir);
|
||||
reportFiles.sort();
|
||||
expect(reportFiles).toEqual(['report-1.zip']);
|
||||
const { exitCode } = await mergeReports(reportDir, { 'PW_TEST_HTML_REPORT_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
const { exitCode } = await mergeReports(reportDir, { 'PLAYWRIGHT_HTML_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
expect(exitCode).toBe(0);
|
||||
|
||||
const htmlReportDir = test.info().outputPath('playwright-report');
|
||||
@ -709,7 +709,7 @@ test('resource names should not clash between runs', async ({ runInlineTest, sho
|
||||
reportFiles.sort();
|
||||
expect(reportFiles).toEqual(['report-1.zip', 'report-2.zip']);
|
||||
|
||||
const { exitCode } = await mergeReports(reportDir, { 'PW_TEST_HTML_REPORT_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
const { exitCode } = await mergeReports(reportDir, { 'PLAYWRIGHT_HTML_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
expect(exitCode).toBe(0);
|
||||
|
||||
await showReport();
|
||||
@ -781,7 +781,7 @@ test('multiple output reports', async ({ runInlineTest, mergeReports, showReport
|
||||
const reportFiles = await fs.promises.readdir(reportDir);
|
||||
reportFiles.sort();
|
||||
expect(reportFiles).toEqual(['report-1.zip']);
|
||||
const { exitCode, output } = await mergeReports(reportDir, { 'PW_TEST_HTML_REPORT_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html,line'] });
|
||||
const { exitCode, output } = await mergeReports(reportDir, { 'PLAYWRIGHT_HTML_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html,line'] });
|
||||
expect(exitCode).toBe(0);
|
||||
|
||||
// Check that line reporter was called.
|
||||
@ -907,7 +907,7 @@ test('onError in the report', async ({ runInlineTest, mergeReports, showReport,
|
||||
const result = await runInlineTest(files, { shard: `1/3` }, { PWTEST_BOT_NAME: 'macos-node16-ttest' });
|
||||
expect(result.exitCode).toBe(1);
|
||||
|
||||
const { exitCode } = await mergeReports(reportDir, { 'PW_TEST_HTML_REPORT_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
const { exitCode } = await mergeReports(reportDir, { 'PLAYWRIGHT_HTML_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
expect(exitCode).toBe(0);
|
||||
|
||||
await showReport();
|
||||
@ -1149,7 +1149,7 @@ test('preserve steps in html report', async ({ runInlineTest, mergeReports, show
|
||||
// relative to the current directory.
|
||||
const mergeCwd = test.info().outputPath('foo');
|
||||
await fs.promises.mkdir(mergeCwd, { recursive: true });
|
||||
const { exitCode, output } = await mergeReports(reportDir, { 'PW_TEST_HTML_REPORT_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'], cwd: mergeCwd });
|
||||
const { exitCode, output } = await mergeReports(reportDir, { 'PLAYWRIGHT_HTML_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'], cwd: mergeCwd });
|
||||
expect(exitCode).toBe(0);
|
||||
|
||||
expect(output).not.toContain('To open last HTML report run:');
|
||||
@ -1326,7 +1326,7 @@ test('keep projects with same name different bot name separate', async ({ runInl
|
||||
await runInlineTest(files('second'), undefined, { PWTEST_BOT_NAME: 'second', PWTEST_BLOB_DO_NOT_REMOVE: '1' });
|
||||
|
||||
const reportDir = test.info().outputPath('blob-report');
|
||||
const { exitCode } = await mergeReports(reportDir, { 'PW_TEST_HTML_REPORT_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
const { exitCode } = await mergeReports(reportDir, { 'PLAYWRIGHT_HTML_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
expect(exitCode).toBe(0);
|
||||
await showReport();
|
||||
await expect(page.locator('.subnav-item:has-text("Passed") .counter')).toHaveText('1');
|
||||
@ -1452,7 +1452,7 @@ test('merge-reports should throw if report version is from the future', async ({
|
||||
await fs.promises.rm(reportZipFile, { force: true });
|
||||
await zipReport(events, reportZipFile);
|
||||
|
||||
const { exitCode, output } = await mergeReports(reportDir, { 'PW_TEST_HTML_REPORT_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
const { exitCode, output } = await mergeReports(reportDir, { 'PLAYWRIGHT_HTML_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
expect(exitCode).toBe(1);
|
||||
expect(output).toContain(`Error: Blob report report-2.zip was created with a newer version of Playwright.`);
|
||||
|
||||
@ -1496,7 +1496,7 @@ test('should merge blob reports with same name', async ({ runInlineTest, mergeRe
|
||||
await fs.promises.cp(reportZip, path.join(allReportsDir, 'report-1.zip'));
|
||||
await fs.promises.cp(reportZip, path.join(allReportsDir, 'report-2.zip'));
|
||||
|
||||
const { exitCode } = await mergeReports(allReportsDir, { 'PW_TEST_HTML_REPORT_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
const { exitCode } = await mergeReports(allReportsDir, { 'PLAYWRIGHT_HTML_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
expect(exitCode).toBe(0);
|
||||
await showReport();
|
||||
await expect(page.locator('.subnav-item:has-text("All") .counter')).toHaveText('10');
|
||||
@ -1887,7 +1887,7 @@ test('preserve static annotations when tests did not run', async ({ runInlineTes
|
||||
`
|
||||
};
|
||||
await runInlineTest(files);
|
||||
const { exitCode } = await mergeReports(reportDir, { 'PW_TEST_HTML_REPORT_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
const { exitCode } = await mergeReports(reportDir, { 'PLAYWRIGHT_HTML_OPEN': 'never' }, { additionalArgs: ['--reporter', 'html'] });
|
||||
expect(exitCode).toBe(0);
|
||||
await showReport();
|
||||
|
||||
|
||||
@ -65,7 +65,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
expect(testInfo.retry).toBe(1);
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html', retries: 1 }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html', retries: 1 }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
await showReport();
|
||||
|
||||
@ -95,6 +95,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
await expect(1).toBe(1);
|
||||
});
|
||||
`,
|
||||
// Note: using PW_TEST_HTML_REPORT_OPEN to test backwards compatibility.
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(1);
|
||||
@ -113,7 +114,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
await expect(page.locator('.attachment-body')).toHaveText(/TESTID=.*/);
|
||||
});
|
||||
|
||||
test('should not throw when PW_TEST_HTML_REPORT_OPEN value is invalid', async ({ runInlineTest, page, showReport }, testInfo) => {
|
||||
test('should not throw when PLAYWRIGHT_HTML_OPEN value is invalid', async ({ runInlineTest, page, showReport }, testInfo) => {
|
||||
const invalidOption = 'invalid-option';
|
||||
const result = await runInlineTest({
|
||||
'playwright.config.ts': `
|
||||
@ -125,7 +126,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
expect(2).toEqual(2);
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: invalidOption });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: invalidOption });
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(1);
|
||||
});
|
||||
@ -143,7 +144,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
testInfo.attachments.push({ name: 'screenshot', path: screenshot, contentType: 'image/png' });
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(1);
|
||||
|
||||
@ -169,7 +170,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
await expect(screenshot).toMatchSnapshot('expected.png');
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.failed).toBe(1);
|
||||
|
||||
@ -243,7 +244,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
await expect.soft(page).toHaveScreenshot({ timeout: 1000 });
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.failed).toBe(1);
|
||||
|
||||
@ -278,7 +279,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
await expect.soft(screenshot).toMatchSnapshot('expected.png');
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.failed).toBe(1);
|
||||
|
||||
@ -309,7 +310,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
await expect.soft(page).toHaveScreenshot({ timeout: 1000 });
|
||||
});
|
||||
`,
|
||||
}, { 'reporter': 'dot,html', 'update-snapshots': true }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { 'reporter': 'dot,html', 'update-snapshots': true }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.failed).toBe(1);
|
||||
|
||||
@ -344,7 +345,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
await expect(screenshot).toMatchSnapshot('expected');
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.failed).toBe(1);
|
||||
|
||||
@ -373,7 +374,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
await expect(true).toBeFalsy();
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.failed).toBe(1);
|
||||
|
||||
@ -404,7 +405,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
await page.evaluate('2 + 2');
|
||||
});
|
||||
`
|
||||
}, {}, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, {}, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(1);
|
||||
|
||||
@ -432,7 +433,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
await expect(true).toBeFalsy();
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.failed).toBe(1);
|
||||
|
||||
@ -452,7 +453,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
await expect(true).toBeFalsy();
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.failed).toBe(1);
|
||||
|
||||
@ -475,7 +476,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
await evaluateWrapper(page, '2 + 2');
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(1);
|
||||
|
||||
@ -508,7 +509,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
await page.evaluate('2 + 2');
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(1);
|
||||
|
||||
@ -538,7 +539,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
await page.evaluate('2 + 2');
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(1);
|
||||
|
||||
@ -562,7 +563,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
await request.dispose();
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(1);
|
||||
|
||||
@ -600,7 +601,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
]);
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(1);
|
||||
|
||||
@ -625,7 +626,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
await page.evaluate('2 + 2');
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(1);
|
||||
|
||||
@ -683,7 +684,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
});
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.passed).toBe(0);
|
||||
|
||||
@ -726,7 +727,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
test.info().annotations.push({ type: 'issue', description: 'I am not interested in this test' });
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(1);
|
||||
|
||||
@ -746,7 +747,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
test.info().annotations.push({ type: 'issue', description: '${server.EMPTY_PAGE}' });
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(1);
|
||||
|
||||
@ -789,7 +790,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
});
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(0);
|
||||
|
||||
await showReport();
|
||||
@ -814,7 +815,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
await testInfo.attach('example.ext with spaces', { body: Buffer.from('b'), contentType: 'madeup' });
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(0);
|
||||
await showReport();
|
||||
await page.getByRole('link', { name: 'passing' }).click();
|
||||
@ -867,7 +868,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
expect('new').toMatchSnapshot('snapshot.txt');
|
||||
});
|
||||
`
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(1);
|
||||
await showReport();
|
||||
await page.click('text="is a test"');
|
||||
@ -894,7 +895,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
expect('newcommon').toMatchSnapshot('snapshot.txt');
|
||||
});
|
||||
`
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(1);
|
||||
await showReport();
|
||||
await page.click('text="is a test"');
|
||||
@ -912,7 +913,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
throw new Error('ouch');
|
||||
});
|
||||
`
|
||||
}, { 'reporter': 'dot,html', 'repeat-each': 3 }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { 'reporter': 'dot,html', 'repeat-each': 3 }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(1);
|
||||
await showReport();
|
||||
|
||||
@ -937,7 +938,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
expect(2).toEqual(2);
|
||||
});
|
||||
`
|
||||
}, { 'reporter': 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { 'reporter': 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(0);
|
||||
await showReport();
|
||||
|
||||
@ -956,7 +957,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
test('sample', async ({}) => { expect(2).toBe(2); });
|
||||
`,
|
||||
'a.spec.js': `require('./inner')`
|
||||
}, { 'reporter': 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { 'reporter': 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
expect(result.exitCode).toBe(0);
|
||||
await showReport();
|
||||
await expect(page.locator('text=a.spec.js')).toBeVisible();
|
||||
@ -997,7 +998,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
await execGit(['commit', '-m', 'awesome commit message']);
|
||||
|
||||
const result = await runInlineTest(files, { reporter: 'dot,html' }, {
|
||||
PW_TEST_HTML_REPORT_OPEN: 'never',
|
||||
PLAYWRIGHT_HTML_OPEN: 'never',
|
||||
GITHUB_REPOSITORY: 'microsoft/playwright-example-for-test',
|
||||
GITHUB_RUN_ID: 'example-run-id',
|
||||
GITHUB_SERVER_URL: 'https://playwright.dev',
|
||||
@ -1043,7 +1044,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
import { test, expect } from '@playwright/test';
|
||||
test('sample', async ({}) => { expect(2).toBe(2); });
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never', GITHUB_REPOSITORY: 'microsoft/playwright-example-for-test', GITHUB_RUN_ID: 'example-run-id', GITHUB_SERVER_URL: 'https://playwright.dev', GITHUB_SHA: 'example-sha' }, undefined);
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never', GITHUB_REPOSITORY: 'microsoft/playwright-example-for-test', GITHUB_RUN_ID: 'example-run-id', GITHUB_SERVER_URL: 'https://playwright.dev', GITHUB_SHA: 'example-sha' }, undefined);
|
||||
|
||||
await showReport();
|
||||
|
||||
@ -1071,7 +1072,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
import { test, expect } from '@playwright/test';
|
||||
test('my sample test', async ({}) => { expect(2).toBe(2); });
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' }, undefined);
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' }, undefined);
|
||||
|
||||
await showReport();
|
||||
|
||||
@ -1095,7 +1096,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
import { test, expect } from '@playwright/test';
|
||||
test('my sample test', async ({}) => { expect(2).toBe(2); });
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
await showReport();
|
||||
|
||||
@ -1177,7 +1178,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
test('pass', ({}, testInfo) => {
|
||||
});
|
||||
`
|
||||
}, { 'reporter': 'html,line' }, { PW_TEST_HTML_REPORT_OPEN: 'never' }, {
|
||||
}, { 'reporter': 'html,line' }, { PLAYWRIGHT_HTML_OPEN: 'never' }, {
|
||||
cwd: 'foo/bar/baz/tests',
|
||||
});
|
||||
expect(result.exitCode).toBe(0);
|
||||
@ -1201,7 +1202,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
test('pass', ({}, testInfo) => {
|
||||
});
|
||||
`
|
||||
}, { 'reporter': 'html,line' }, { 'PW_TEST_HTML_REPORT_OPEN': 'never', 'PLAYWRIGHT_HTML_REPORT': '../my-report' }, {
|
||||
}, { 'reporter': 'html,line' }, { 'PLAYWRIGHT_HTML_OPEN': 'never', 'PLAYWRIGHT_HTML_OUTPUT_DIR': '../my-report' }, {
|
||||
cwd: 'foo/bar/baz/tests',
|
||||
});
|
||||
expect(result.exitCode).toBe(0);
|
||||
@ -1250,7 +1251,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
expect(1).toBe(2);
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.passed).toBe(3);
|
||||
@ -1324,7 +1325,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
expect(1).toBe(1);
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(3);
|
||||
@ -1366,7 +1367,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
expect(1).toBe(1);
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(3);
|
||||
@ -1407,7 +1408,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
expect(1).toBe(1);
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(3);
|
||||
@ -1445,7 +1446,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
});
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.passed).toBe(2);
|
||||
@ -1523,7 +1524,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
});
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(5);
|
||||
@ -1566,7 +1567,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
expect(1).toBe(2);
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.passed).toBe(1);
|
||||
@ -1615,7 +1616,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
});
|
||||
}
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.passed).toBe(7);
|
||||
@ -1689,7 +1690,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
expect(1).toBe(2);
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.passed).toBe(2);
|
||||
@ -1755,7 +1756,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
expect(1).toBe(2);
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.passed).toBe(2);
|
||||
@ -1804,7 +1805,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
expect(1).toBe(2);
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.passed).toBe(1);
|
||||
@ -1882,7 +1883,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
expect(1).toBe(1);
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(3);
|
||||
@ -2021,7 +2022,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
expect(1).toBe(0);
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.passed).toBe(3);
|
||||
@ -2110,7 +2111,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
test('passes', () => {});
|
||||
}
|
||||
`,
|
||||
}, { reporter: 'html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
await showReport();
|
||||
|
||||
@ -2139,7 +2140,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
});
|
||||
test('test 6', async ({}) => {});
|
||||
`,
|
||||
}, { reporter: 'html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
await showReport();
|
||||
|
||||
@ -2167,7 +2168,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
test('b test 1', async ({}) => {});
|
||||
test('b test 2', async ({}) => {});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(4);
|
||||
@ -2197,7 +2198,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
test('failed title', async ({}) => { expect(1).toBe(1); });
|
||||
test('passes title', async ({}) => { expect(1).toBe(2); });
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.passed).toBe(1);
|
||||
@ -2220,7 +2221,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
test('test1', async ({}) => { expect(1).toBe(1); });
|
||||
test('test2', async ({}) => { expect(1).toBe(2); });
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.passed).toBe(1);
|
||||
@ -2257,7 +2258,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
expect(1).toBe(1);
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(1);
|
||||
@ -2286,7 +2287,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
expect(1).toBe(1);
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(1);
|
||||
@ -2315,7 +2316,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
expect(1).toBe(1);
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(1);
|
||||
@ -2344,7 +2345,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
expect(1).toBe(1);
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(1);
|
||||
@ -2374,7 +2375,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
'playwright.config.ts': `
|
||||
export default { globalTeardown: './globalTeardown.ts' };
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.passed).toBe(1);
|
||||
@ -2395,7 +2396,7 @@ for (const useIntermediateMergeReport of [false] as const) {
|
||||
});
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'dot,html' }, { PW_TEST_HTML_REPORT_OPEN: 'never' });
|
||||
}, { reporter: 'dot,html' }, { PLAYWRIGHT_HTML_OPEN: 'never' });
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.passed).toBe(1);
|
||||
|
||||
@ -281,7 +281,7 @@ test.describe('report location', () => {
|
||||
test('pass', ({}, testInfo) => {
|
||||
});
|
||||
`
|
||||
}, { 'reporter': 'json' }, { 'PW_TEST_HTML_REPORT_OPEN': 'never', 'PLAYWRIGHT_JSON_OUTPUT_NAME': '../my-report.json' }, {
|
||||
}, { 'reporter': 'json' }, { 'PLAYWRIGHT_JSON_OUTPUT_NAME': '../my-report.json' }, {
|
||||
cwd: 'foo/bar/baz/tests',
|
||||
});
|
||||
expect(result.exitCode).toBe(0);
|
||||
@ -302,7 +302,7 @@ test.describe('report location', () => {
|
||||
test('pass', ({}, testInfo) => {
|
||||
});
|
||||
`
|
||||
}, { 'reporter': 'json' }, { 'PW_TEST_HTML_REPORT_OPEN': 'never', 'PLAYWRIGHT_JSON_OUTPUT_FILE': '../my-report.json' }, {
|
||||
}, { 'reporter': 'json' }, { 'PLAYWRIGHT_JSON_OUTPUT_FILE': '../my-report.json' }, {
|
||||
cwd: 'foo/bar/baz/tests',
|
||||
});
|
||||
expect(result.exitCode).toBe(0);
|
||||
|
||||
@ -70,7 +70,7 @@ for (const useIntermediateMergeReport of [false, true] as const) {
|
||||
});
|
||||
});
|
||||
`,
|
||||
}, { reporter: 'list' }, { PW_TEST_DEBUG_REPORTERS: '1', PW_TEST_DEBUG_REPORTERS_PRINT_STEPS: '1', PLAYWRIGHT_FORCE_TTY: '80' });
|
||||
}, { reporter: 'list' }, { PW_TEST_DEBUG_REPORTERS: '1', PLAYWRIGHT_LIST_PRINT_STEPS: '1', PLAYWRIGHT_FORCE_TTY: '80' });
|
||||
const text = result.output;
|
||||
const lines = text.split('\n').filter(l => l.match(/^\d :/)).map(l => l.replace(/[.\d]+m?s/, 'Xms'));
|
||||
lines.pop(); // Remove last item that contains [v] and time in ms.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user