Commit Graph

138 Commits

Author SHA1 Message Date
Simon Backx
f568b35f26 Fixed 2023 snapshot tests
no issue

Updated the snapshot files for the email preview test snapshots that contain 2023 in the email footer.
2023-01-02 11:18:07 +01:00
Simon Backx
0825a2d7f4
🐛 Fixed importing existing member resetting newsletters (#16017)
fixes https://github.com/TryGhost/Team/issues/2386

**Issue:**
- When trying to import a member that already exists, and has
'subscribed' set to 'true' in the CSV, the newsletters the member is
subscribed to are reset to the default newsletters.
- When ediging a member with the API and setting `subscribed` to true,
the same happens.

**Cause:**
A faulty check for the `status` property of a newsletter.

Fixed and added a new E2E test.
2022-12-15 17:45:11 +01:00
Naz
879aad263d
Added email verification trigger test
refs https://github.com/TryGhost/Toolbox/issues/476

- The email verification trigger and host settings related bugs have been a cause of bugs in past releases.  The admin client verification source did not have any test coverage in the past.
- The members test suite size is getting out of hand. This test is quite verbose, because of the state it's trying to check.
- In the future we should consider splitting up Member API (and probably other) test suites into smaller pieces.
2022-12-05 18:24:28 +07:00
Naz
e170f293e3
Extracted sleep method to e2e framework module
no issue

- The sleep method has been used in 8 modules reimplementing the same thing over and over again. It's usually a sign of async event processing outside of the request/response loop. It's good to have a single point of implementation for a "hack" like this, so we could track it easier and address the even processing delay in a more optimal way centrally if it ever becomes a bottleneck
2022-12-05 17:26:29 +07:00
Simon Backx
2a3a72e4d5
Added email debug API (#15915)
fixes https://github.com/TryGhost/Team/issues/2346

- Adds email batch browse endpoint
- Adds email recipient failures browse endpoint
- Adds new fixtures and E2E tests for the new API
- Added support for snapshot tests to have 'nullable' types.
2022-12-02 09:44:32 +01:00
Simon Backx
4c166e11df
Added E2E tests for batch sending (#15910)
refs https://github.com/TryGhost/Team/issues/2339

- Includes a new pattern in the job manager that allows us to properly
await jobs.
- Added new convenience mocking methods to stub settings
- Tests the main flows for bulk sending:
    - Sending in multiple batches
    - Sending to multiple segments
    - Handling a failed batch and retrying that batch
- Fixes bug in batch generation (ordering not working)

In a different PR I'll add more detailed tests.
2022-12-01 13:43:49 +01:00
Naz
80decf8303
Fixed invalid query strings in tests
closes https://github.com/TryGhost/Team/issues/2324

- It seemed like the "limit" query parameter did not work properly returning multiple entries from the endpoint. In reality the whole query string was ignored because of an error in the "filter" part of the query ^_^
2022-11-30 13:31:06 +07:00
Elena Baidakova
1b784b5ec5
Added events to Post Analytics page (#15886)
closes TryGhost/Team#2313
- Added Sent event to Post analytics and Members feed. Now post can be
Sent or Received or Bounced.
- Excluded Delivered event from Sent filter on backend.
2022-11-28 17:43:35 +04:00
Naz
22738b1b50 🔒 Disabled editable relations by default
refs https://github.com/TryGhost/Ghost/security/advisories/GHSA-9gh8-wp53-ccc6
refs https://github.com/TryGhost/Toolbox/issues/465

- Bookshelf relations allows us to edit relational records by default, which was used liberally in the codebase.
- Not having a clear track record of editable relations left the model layer prone to triggering unwanted nested saves and created a vulnerability where members were able to edit newsletter settings.
- With explicit editable relations it's easier to keep track of relations having editable access to related records. Makes the relational data modification pattern safer to use too.
- Anyone running 5.x should update to 5.24.1

Credits: Dave McDaniel and other members of [Cisco Talos](https://talosintelligence.com/vulnerability_reports)
2022-11-28 18:39:39 +07:00
Simon Backx
4b4592630f
Added new email batch sending service (#15865)
fixes https://github.com/TryGhost/Team/issues/2284

New batch sending flow (still WIP). Logs the sent emails instead of actually sending them. Unit tests are coming in later commits.
2022-11-23 11:33:44 +01:00
Rishabh
5780fc2a93 Added new source and source type columns to emails table
refs https://github.com/TryGhost/Team/issues/2280

We are moving away from storing html and plaintext on email and instead will store the email data in source and source_type columns which allows us to store the email in other formats like mobiledoc and lexical. Storing in those formats allows greater flexibility for later html generation

- adds new `source` column that stores `mobiledoc`/`lexical`/`html` data for a newsletter
- adds new `source_type` column that stores one of `mobiledoc`/`lexical`/`html` to identify type of source
2022-11-23 15:04:11 +05:30
Simon Backx
44f189b56a
Added email service package (#15849)
fixes https://github.com/TryGhost/Team/issues/2282

Added a new email service package that is used when the email stability
flag is enabled. Currently not yet implemented so will throw an error
for all entry points (if flag enabled).

Removed usage of `labs.isSet.bind` across the code, because that breaks
the stubbing of labs by `mockManager.mockLabsEnabled` and
`mockManager.mockLabsDisabled`. `flag => labs.isSet(flag)` should be
used instead.

All email depending tests now disable the `emailStability` feature flag
to keep the tests passing + make sure we still run all the tests for the
old flow while the email stability package is being built.
2022-11-21 10:29:53 +01:00
Fabien 'egg' O'Carroll
8283de99c8
Wired up EmailSuppressionList to Members Admin API (#15848)
refs https://github.com/TryGhost/Team/issues/2268

The approach of using the service to lead email suppression data as
opposed to bookshelf relations allows us to wire things up without
having implemented the database. The getBulkSuppressionData allows us to
do this without much of a DB performance hit.
2022-11-18 16:28:13 +07:00
Ronald Langeveld
08232bd538
Cleaned up GA Flags (#15762)
closes https://github.com/TryGhost/Team/issues/2126

- Cleaned up the following GA flags: `newsletterPaywall`, `freeTrial`, `compExpiring`, `searchHelper`, `emailAlerts`, `fixNewsletterLinks`.
2022-11-18 16:09:25 +07:00
Ronald Langeveld
e0787b4e83
Added specific newsletter support for bulk unsubscribes (#15742)
closes https://github.com/TryGhost/Team/issues/2013

Added support to bulk unsubscribe a selected (filtered) list on members from specific, selected newsletters.
2022-11-16 14:29:00 +07:00
Sodbileg Gansukh
78bff39c23
Update the cover image in default fixtures (#15817)
- updated the cover image to be simpler
- made the change in text fixtures as well, just to keep the fixtures in sync

Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
2022-11-15 21:19:50 +00:00
Simon Backx
cdccf5c46a
Reduced default post relations (#15798)
fixes https://github.com/TryGhost/Team/issues/2085

Don't load relations we don't need anymore for the posts table. And
reload the individual post when we open the analytics page with more
relations that we actually need.
2022-11-15 10:17:26 +01:00
Ronald Langeveld
6214812ac0
Added e2e test for bulk member deletion (#15730)
ref https://github.com/TryGhost/Team/issues/1446

- tests the `delete` api endpoint method for deleting a bulk set of members.
2022-11-08 17:02:55 +07:00
Simon Backx
231ceedc72
Readded activity feed pagination tests (#15776)
fixes https://github.com/TryGhost/Team/issues/2218

The test should run a lot faster now.
2022-11-07 15:08:56 +01:00
Fabien 'egg' O'Carroll
a0ebb9a6f3
🐛 Fixed Tier description not being set (#15741)
closes https://github.com/TryGhost/Ghost/issues/15740

The validation function for a Tier description was not returning the
validated value, which meant we were unable to set the Tier
description.
2022-11-01 16:25:55 +07:00
Hannah Wolfe
5fe6687c2a
Skipped slow activity feed tests temporarily
- These tests are very slow, and make the build fail about 2/3 times
- Temporarily skipping until we can fix, as I want to get all our outstanding hacktoberfest PRs merged
2022-10-30 14:52:30 +00:00
Simon Backx
13fd64ebf7 Fixed tests for flag bumps and new setting
no issue
2022-10-27 18:26:46 +02:00
Simon Backx
b916300ceb
Added aggregated click events (#15713)
fixes https://github.com/TryGhost/Team/issues/2175

- New event type `aggregated_click_event` that is disabled by default in all the existing activity feeds
- This returns click events, but only the first click events for each member/post combination.
- It includes the total count of unique link clicks for that member on that post combination
- Had to resort to some custom knex queries to make this work easily
- Requires `@tryghost/bookshelf-pagination@0.1.31`, included in `@tryghost/bookshelf-plugins@0.6.1` (this fixes an issue with custom selects breaking the total count query of pages)
- Went a bit overboard with the pagination tests to cover as much unknown edge cases as possible
2022-10-27 17:23:45 +02:00
Simon Backx
b911208b41
Improved filter support in activity API to allow pagination (#15684)
fixes https://github.com/TryGhost/Team/issues/2129

- This changes how the activity feed API parses the filter.
- We now parse the filter early to a MongoDB filter, and split it in two. One of the filters is applied to the pageActions, and the other one is used individually for every event type. We now allow to use grouping and OR's inside the filters because of this change. As long as we don't combine filters on 'type' with other filters inside grouped filters or OR, then it is allowed.
- We make use of mongoTransformer to manually inject a mongo filter without needing to parse it from a string value again (that would make it a lot harder because we would have to convert the splitted filter back to a string and we currently don't have methods for that).
- Added sorting by id for events with the same timestamp (required for reliable pagination)
- Added id to each event (required for pagination)
- Added more tests for filters
- Added test for pagination
- Removed unsued getSubscriptions and getVolume methods

Used new mongo utility methods introduced here: https://github.com/TryGhost/NQL/pull/49
2022-10-27 12:13:24 +02:00
Simon Backx
076e3c02b2
Added linking between member and subscription created events (#15693)
fixes https://github.com/TryGhost/Team/issues/2160

- Adds a `batch_id` to both events that contain the same ID if they were created at the same time.
- Removes duplicate signup/conversion events using the batch_id
- Requires an update in mongo-knex to work (refs https://ghost.slack.com/archives/C02G9E68C/p1666773313272409?thread_ts=1666767872.375009&cid=C02G9E68C)
- Some dependencies needed an update to load the latest mongo-knex
- Added tiers to membersUtils, loaded on startup (we can start to use this instead of fetching it every time)
2022-10-27 11:44:19 +02:00
Naz
6bd2769146
Hardened tiers-related snapshot tests
refs https://github.com/TryGhost/Team/issues/2077

- Members and Posts test suites were using a broad tiers property matcher, which is an anti-pattern for snapshot tests. Without more specific snapshots it would be very hard to track down tier-related breaking changes!
- This change is groundwork for a refactor coming in tier usage at API's output serializers
2022-10-27 15:54:28 +08:00
Elena Baidakova
e3ab868b83
Added email feedback column (#15698)
closes TryGhost/Team#2159
- Added column to email table
- Hide the feedback tab on frontend depending on the column value

Co-authored-by: Daniel Lockyer <daniellockyer@fastmail.com>
2022-10-27 11:22:50 +04:00
Elena Baidakova
57817eefc8
Added feedback buttons to emails (#15695)
closes TryGhost/Team#2075
2022-10-26 19:19:05 +04:00
Fabien "egg" O'Carroll
25d8d694a0 Wired up the payment service to create stripe checkout sessions
refs https://github.com/TryGhost/Team/issues/2078
2022-10-25 09:03:04 +07:00
Fabien "egg" O'Carroll
31610f9b94 Wired up Tiers service to Tiers Content & Admin API
refs https://github.com/TryGhost/Team/issues/2078
2022-10-25 09:02:59 +07:00
Simon Backx
fd91f7eebb
Added email sent events (#15682)
fixes https://github.com/TryGhost/Team/issues/2137

For the analytics page, we need the sent events to show up immediately
after sending an email. Otherwise we need to wait for emails to be
marked as received (which takes too long) before being able to show them
on the analytics page.

This adds the email_sent_event, which is hidden by default everywhere
and used on the analytics page.
2022-10-24 11:11:44 +02:00
Elena Baidakova
9720459f55
Added pie chart for activity feed (#15673)
closes TryGhost/Team#2088
- Added pie chart to feedback event
- Added `negative_feedback` field to response from BE
2022-10-21 21:34:05 +04:00
Rishabh Garg
05330316e3
Allowed fixing newsletter links (#15672)
refs https://github.com/TryGhost/Team/issues/2116

- allows site owners to edit a link in a post that has already been sent out, fixing any typos or other mistakes
- resets click counter for the edited link back to 0 so site owners can see the clicks on new link, doesn't change the overall click count
2022-10-21 19:38:57 +05:30
Naz
b552727b69
Changed members export to contain tiers
refs https://github.com/TryGhost/Team/issues/1076

- The "products" is a legacy term for what is now "tiers" since multiple tiers feature introduction in https://github.com/TryGhost/Ghost/releases/tag/v4.39.0
- Note, the "tiers" is a field meant for informational purposes and cannot be imported back into Ghost site.
2022-10-21 18:58:59 +08:00
Rishabh
ba6fe63250 Fixed cached redirects for edited newsletter links
refs https://github.com/TryGhost/Team/issues/2135

The email link redirects on Pro are cached as 302 redirects in Varnish, so we're missing further clicks after the first one for each member, until the cache is invalidated. This change invalidates cache on link edits to ensure that we correctly redirect members to updated link everytime
2022-10-21 16:07:34 +05:30
Rishabh
d7b3721036 Added edited property to links api
refs https://github.com/TryGhost/Team/issues/2104

- adds a boolean `edited` property to links api that denotes if the link has been edited
2022-10-20 18:17:44 +05:30
Rishabh
d8bacf12d1 Added endpoint for fixing newsletter links
refs https://github.com/TryGhost/Team/issues/2104

- adds new bulk edit endpoint for links, updates all matching link with the current redirect url and update to new url
2022-10-20 17:50:02 +05:30
Simon Backx
adf10f0e76
Added support for filtering email events by post_id (#15666)
refs https://github.com/TryGhost/Team/issues/2093
2022-10-20 13:29:00 +02:00
Simon Backx
6380b82793
Added sentiment ordering and include for posts (#15657)
fixes https://github.com/TryGhost/Team/issues/2090

- This changes how sentiment is exposed in the API. Now it is exposed as a `sentiment` relation, directly on the model (no longer in counts). Internally we still use `count.sentiment`.
- Content API users (and themes) can include the 'sentiment' relation and order by sentiment.
- Updated Admin to use sentiment instead of count.sentiment
2022-10-19 16:50:58 +02:00
Simon Backx
a01fb5f1aa
Added post_id filter and total to activity feed API (#15650)
fixes https://github.com/TryGhost/Team/issues/2091
fixes https://github.com/TryGhost/Team/issues/2089

- Added new fixtures to make testing easier for the activity feed
- Improved E2E test coverage of activity feed with separate test file
- Added data.post_id filter to enable filtering by events related to a
given post
- Fixed return types in JSDoc of test agents (TypeScript interprets
these as `typeof Agent` if we don't add `InstanceType<Agent>`)
- Added total pagination metadata to activity feed API (to allow a basic
type of pagination using filters)
2022-10-18 15:52:04 +02:00
Simon Backx
22fe1c01de
Added conversions count and separate analytics page (#15637)
fixes https://github.com/TryGhost/Team/issues/2084

- When audience feedback is enabled, we use a single 'conversions' count instead of having separate ones for signups and paid conversions.
- The analytics component is separated so we can change it without breaking the existing page.
2022-10-17 13:02:39 +02:00
Elena Baidakova
e831be6bc2
Added the feedback buttons in the emails (#15619)
closes TryGhost/Team#2046
closes TryGhost/Team#2045
- Added feedback buttons markup.
- Added feedback links generation.
2022-10-14 18:12:17 +04:00
Fabien 'egg' O'Carroll
bd0f4b4b8c
Added Tier price and currency data to products table (#15366)
refs https://github.com/TryGhost/Team/issues/1765

In order to better handle deleted objects in Stripe we want to decouple
Members from Stripe.

These changes allow us to have the Tier concept completely independent
of the Stripe tables, such that the Stripe data can be generated as/when
it's needed - which will help to protect against missing data.
2022-10-14 06:40:17 +01:00
jbenezech
75811f35bc
🐛 Fixed embedded cards for non-utf8 content (#15578)
closes: https://github.com/TryGhost/Ghost/issues/14973

- When fetching content using a non-standard charset, characters were notproperly decoded to utf-8 resulting in mangled text in the editor -> Detect charset and use iconv to decode the page text

- When requesting a non bookmark card, if no oembed data could be foundand we fallback to bookmark, a second network request to fetch the content was issued. This seemed unnecessary -> refactored to avoid that
2022-10-13 12:19:47 +01:00
Daniel Lockyer
e2ba19b0df
Fixed import of bson-objectid in accordance to the typings
- there's a few different ways we can import it but I've chosen to
  append `.default` as we have done in several other places in the code
2022-10-12 14:54:35 +07:00
Simon Backx
68bdc1afea
Added post sentiment (#15592)
fixes https://github.com/TryGhost/Team/issues/2054

This change adds the sentiment and positive_feedback counts to the posts models. This change isn't really ideal because there are some problems here:
- sentiment isn't really a count
- we don't need to include the sentiment and positive_feedback as a default for posts (but the same is true for attribution)

It would make sense to move this to separate endpoints that only fetch the analytics for a given post when the analytics page is opened. But for our initial skateboard version of audience feedback this should be a good start to already see the data.
2022-10-11 17:52:14 +02:00
Elena Baidakova
1221ba5d1d
Added feedback_enabled to newsletters table (#15589)
closes TryGhost/Team#2042
- Added ability to enable audience feedback per newsletter (just on BE side).
2022-10-11 16:06:26 +04:00
Naz
714e108d40
Fixed typo 2022-10-11 17:24:11 +08:00
Naz
9b34bd70a2
Added test coverage for Subscription edit API
refs https://github.com/TryGhost/Team/issues/2047

- We anticipate upcoming changes in the PUT /members/:id/subscriptions/:subscription_id endpoint , so covered it with a snapshot test to track the differences more precisely.
- Note, the test case contains a more explicit outgoing HTTP request mocking.
2022-10-11 17:24:00 +08:00
Simon Backx
7e3b41f643
Removed emailClicks feature flag (#15556)
fixes https://github.com/TryGhost/Team/issues/2028

Since link clicks became GA, some older components and templates are no longer used.
2022-10-07 14:27:57 +02:00