playwright/utils/flakiness-dashboard
Max Schmitt f518684d89
chore: fix parameterized tests on the flakiness dashboard (#27900)
Abstract issue: I have parameterised tests via the titlePath (describe)
and they are not shown on the flakiness dashboard. Before #24486 only
title() was shown, after #24486 titlePath() will be displayed, but we
still did have the testId based on the test.title() and not
test.titlePath(). This ends up that they will still be treated as a
single spec.

After this change they are not treated as a single spec anymore and
treated as different ones:

<img width="891" alt="image"
src="https://github.com/microsoft/playwright/assets/17984549/f24284cd-5d94-4f7e-a45d-8c8e5eb537ef">

Note: This is tested.

Follow-up on https://github.com/microsoft/playwright/pull/24486.
2023-11-01 17:28:30 +01:00
..
processing chore: fix parameterized tests on the flakiness dashboard (#27900) 2023-11-01 17:28:30 +01:00
.gitignore devops: add source code for flakiness dashboard (#4479) 2020-11-23 15:02:04 -08:00
host.json chore: update flakiness-dashboard extension bundle (#20291) 2023-01-23 06:14:21 -08:00
package-lock.json chore: fix parameterized tests on the flakiness dashboard (#27900) 2023-11-01 17:28:30 +01:00
package.json chore: fix parameterized tests on the flakiness dashboard (#27900) 2023-11-01 17:28:30 +01:00
README.md chore: fix parameterized tests on the flakiness dashboard (#27900) 2023-11-01 17:28:30 +01:00

Flakiness Dashboard Backend

This directory contains source code for the Azure function that we use to aggregate test reports. The data is consumed by https://devops.playwright.dev/flakiness.html

Publish

Azure Functions Core Tools is not available on macOS M1 yet, so we use GitHub Codespaces to publish the function.

Via GitHub Codespaces:

  • Install Azure Functions Core Tools version 4:
    curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.gpg
    mv microsoft.gpg /etc/apt/trusted.gpg.d/microsoft.gpg
    sh -c 'echo "deb [arch=amd64] https://packages.microsoft.com/repos/microsoft-ubuntu-$(lsb_release -cs)-prod $(lsb_release -cs) main" > /etc/apt/sources.list.d/dotnetdev.list'
    apt-get update && apt-get install azure-functions-core-tools-4
    
  • Install Azure CLI:
    curl -sL https://aka.ms/InstallAzureCLIDeb | bash
    
  • Login to Azure:
    az login --use-device-code
    
  • Install NPM Deps (node_modules/ folder will be published as-is):
    cd utils/flakiness-dashboard/
    npm install
    
  • Publish:
    func azure functionapp publish folio-flakiness-dashboard --javascript