refs: https://github.com/TryGhost/DevOps/issues/78
Re-introduce parallel browser tests
These were adding in a previous PR, but the difference between local
running tests and using CI introduced failures.
Added additional fixes to ensure the Stripe API key is used in the CLI when running in CI.
refs: https://github.com/TryGhost/DevOps/issues/78
Instead of running a single instance of Ghost, we now run an instance of Ghost for each test worker.
This has the unfortunate effect that a test failing will close and restart a new instance of Ghost, but in general will be multiple times faster than sequential execution of tests.
fixes https://github.com/TryGhost/Product/issues/3723
This also fixes usage of localhost instead of 127.0.0.1 as a test URL
for playwright. This caused issues for cookies because the member
impersonation navigated to 127.0.0.1 instead of localhost, meaning that
the next page.goto call would go to localhost and lose the cookies.
refs TryGhost/Team#2833
- for mocha tests, we can add `this.retries(1)` to any flaky tests
- for playwright tests, we can add `test.describe.configure({ retries:
1})` to any `describe` block
- not a long-term solution, but it should help mitigate issues with flaky
tests in short term
refs TryGhost/Team#2667
- Added notification check before checking that user successfully unsubscribed from all newsletters. It helps to make the test more stable
refs https://github.com/TryGhost/Team/issues/2371
- cleans up and adds comments for portal playwright tests
- updates data test attributes for portal trigger and popup selectors for consistency
- updates data attribute usage for offers
- adds test that cover creating and signing up to multiple-month/forever offers
- checks that the offer information is shown to members during signup and in account detail
closes https://github.com/TryGhost/Team/issues/2376#event-8026429598
- if an offer is expired/in past, we no longer show it in member account info against the price
- one-time offers are never showed in portal in member account detail, as the payment information shown to member in Portal points to charge at next payment
- if trial days are over for a subscription, portal doesn't show any offer data on member account detail
refs https://github.com/TryGhost/Team/issues/2371
- playwright tests were broken due to state changes based on prev tests that were not accounted for
- in case of multiple newsletters, portal tests expected another step between stripe checkout for newsletter selection
- site settings test was disabling members, but not re-enabling it back
refs TryGhost/Team#2371
- check that members can unsubscribe from newsletters by toggling
preferences in their account settings
- check that member can log out
refs
f5aae1e2c5
refs
0f9ed54a6f
- changing playwright portal tests to work for single tier setup caused failure for comped upgrade tests as they were relying on button text that changed
refs. https://github.com/TryGhost/Team/issues/2371
- two extra assertion was needed for discount and free-trials to check
if the offers are listed in ‘Active’ offers and the URLs load portal
refs. https://github.com/TryGhost/Team/issues/2371
- deleteAllMembers was an unnecessary step
- since there's a generated code appended to the name of the archived offer, it had to be shorter to avoid potential naming conflicts
refs https://github.com/TryGhost/Team/issues/2371
Note that the "Choose" button is "Continue" when running this test
standalone so currently it needs to run with the full suite.
refs. https://github.com/TryGhost/Team/issues/2371
- Test for archived offers should be moved to ‘Archived’ view of the offer list in Admin, and the offer URL should redirect to the site's homepage for logged out visitors
refs https://github.com/TryGhost/Team/issues/2371
- tests that a free member can upgrade to a paid tier via stripe checkout and the payment details are reflected in portal and member detail page on admin
refs https://github.com/TryGhost/Team/issues/2369
- this checks whether the Offer redemption count is set to 1, which
would be indicative that the Offer was successfully counted as
redeemed
- we should start to keep tests grouped by their area, so first we split
by Admin tests and then Portal tests, and within that we split into
setup/Tiers/Offers etc
