Commit Graph

12082 Commits

Author SHA1 Message Date
Kevin Ansfield
9586d1ce56 Fixed sqlite storing member email open rate as floats
no issue

- sqlite will store a float in an integer column due to it's type affinity resulting in long decimal numbers in the UI when we're expecting an integer
- use the `ROUND()` function to ensure we're storing integers in place of floats when performing open rate average calculations
2020-12-10 10:00:25 +00:00
Renovate Bot
c2bb7bd106 Update dependency @tryghost/mw-session-from-token to v0.1.12 2020-12-10 09:43:25 +00:00
Renovate Bot
7b4e041203 Update dependency @tryghost/adapter-manager to v0.2.4 2020-12-10 09:43:13 +00:00
Renovate Bot
89f77c8248 Update dependency @tryghost/job-manager to v0.5.0 2020-12-10 09:43:04 +00:00
Renovate Bot
0e6ee237ac Update dependency @tryghost/session-service to v0.1.13 2020-12-10 09:42:48 +00:00
Renovate Bot
712f78bc24
Update dependency @tryghost/helpers to v1.1.35 2020-12-09 21:48:48 +00:00
Kevin Ansfield
24bd2d78cb Fixed validation errors when saving members
no issue

- schema validation was erroring due extra properties on the member resource that were recently added to the output causing the client to send them back when saving
- updated @tryghost/admin-api-schema to include skips for the new read-only properties
2020-12-09 21:47:14 +00:00
Fabien O'Carroll
32fe260763 Updated canary members controller to use new Importer
no-issue

This completely replaces the old import functionality with the new
importer!
2020-12-09 19:25:57 +00:00
Fabien O'Carroll
ff12d7a89a Updated @tryghost/members-{api,csv} for importer
no-issue

- Updates members-api to throw errors when attempting to link to
  stripe customers if it is not configured, which allows us to include
  the errors in the CSV

- Updates members-csv to handle the header mappings as well as parsing
  the values so that we can read a CSV file and get back valid member
  objects
2020-12-09 19:25:57 +00:00
Fabien O'Carroll
cfad396652 Exported MembersCSVImporter from Members service
no-issue

This will allow the controller to be updated to use the new importer
2020-12-09 19:25:57 +00:00
Fabien O'Carroll
ff7773d1cf Added new members importer module
no-issue

This module encapsulates the work around performing imports, it
currently uses the concept of a "Job" which at the moment is not
persisted to the database, however when we want to look at resuming
imports after a server restart, this should give us the flexibility to
do it.
2020-12-09 19:25:57 +00:00
Fabien O'Carroll
c7359a083f Added email template for completion of background jobs
no-issue

This will be used to generate email content used for notifying users
that their import has been completed.
2020-12-09 19:25:57 +00:00
Fabien O'Carroll
73dc19e498 Removed Members CSV import validation endpoint
no-issue

The new import flow does not use prevalidation of the CSV file, so we
have no use for this anymore.
2020-12-09 19:25:57 +00:00
Fabien O'Carroll
83c5270217 Removed support for batched CSV importer
no-issue

We are rewriting the Members CSV importer to use background jobs, the
batched importer will no longer be used locally.
2020-12-09 19:25:57 +00:00
Kevin Ansfield
baf635e27b Updated email stats aggregator for new member count columns
refs https://github.com/TryGhost/Ghost/issues/12461

- added two default aggregations for overall email count and opened email count
- when number of tracked emails is sufficient add the open rate aggregation to the update query
2020-12-09 13:14:39 +00:00
Kevin Ansfield
b1aafd715d
Added members.email_{count,opened_count} column migrations (#12470)
refs https://github.com/TryGhost/Ghost/issues/12461

- adds `members.email_count` and `members.email_opened_count` columns to contain cached counts for faster queries when outputting member data via API
- adds migration to populate cached counts with existing data
  - tested locally on ~50k members which took ~4sec on mysql
- updates members output serializer to include the new fields in API output
2020-12-09 12:21:56 +00:00
Thibaut Patel
c25344d414
Refactored core/server/lib/image for Dependency Injection (#12463)
no issue
2020-12-09 13:19:22 +01:00
Renovate Bot
e91f740ade Update dependency jwks-rsa to v1.12.0 2020-12-09 09:57:24 +00:00
Renovate Bot
3d4f5b5ee5
Update dependency uuid to v8.3.2 2020-12-08 21:23:14 +00:00
Kevin Ansfield
748a8e0d0f
Added email_open_rate order option to members API (#12462)
refs https://github.com/TryGhost/Ghost/issues/12421

- add `orderRawQuery` function to members model so that we can ensure members with an open rate are ordered before members without an open rate no matter the order direction chosen
- added `email_open_rate` to members in the test fixtures to allow testing of order
2020-12-08 12:43:48 +00:00
Kevin Ansfield
567eb6325f
Added members.email_open_rate aggregation to email analytics (#12458)
refs https://github.com/TryGhost/Ghost/issues/12421
requires https://github.com/TryGhost/Ghost/pull/12457

- updates stats aggregator to calculate and store an open rate for each member
  - uses two queries because I couldn't find a reasonable approach to perform the update in a single query as per the email aggregation
  - benchmarked locally at <1sec/1000members
  - will not store an open rate unless the number of tracked emails sent to a member is above a certain threshold (defaults to 5) to avoid new members being heavily weighted
- fixes typo in EmailAnalytics that was stopping member stats from being aggregated
2020-12-08 12:43:10 +00:00
Kevin Ansfield
9fd6f30fd7
Added migration for members.email_open_rate column (#12457)
refs https://github.com/TryGhost/Ghost/issues/12421

- nullable so we can distinguish between members that have and haven't received any trackable emails
- indexed because we'll be using this column for sorting
2020-12-08 12:20:17 +00:00
Daniel Lockyer
99632eab43
Merged 3.39.3 into master
v3.39.3

* tag '3.39.3':
  v3.39.3
  Updated Ghost-Admin to v3.39.3
2020-12-08 09:42:01 +00:00
Daniel Lockyer
6ca434a16f
v3.39.3 2020-12-08 09:37:00 +00:00
Daniel Lockyer
894d18c641
Updated Ghost-Admin to v3.39.3 2020-12-08 09:37:00 +00:00
renovate[bot]
85f94ad4b4
Update dependency @tryghost/job-manager to v0.4.0 (#12460)
Co-authored-by: Renovate Bot <bot@renovateapp.com>
2020-12-08 17:48:13 +13:00
Kevin Ansfield
5d11475f32 Fixed false positive image-size tests
no issue

- fixed usage of rewire and sinon spies
  - spies actually set (and reset) on the rewired module so the spies are called in place of the original functions
- fixed should syntax for spies, they were written with chai syntax but we don't use chai in server-side code
- chained `.catch(done)` to the `.catch(assertions)` in tests that are testing errors so that we don't have unhandled rejections and 60s timeouts when an assert fails
- changed `.gif` to `.ico` to match current fetch-only extension list
- fixed "[failure] can handle an error with statuscode not 200 (image-size)" which was expecting the `sizeOf` function to be called when it shouldn't be called
2020-12-07 22:38:14 +00:00
Renovate Bot
3aa43eb36f
Update dependency js-yaml to v3.14.1 2020-12-07 19:50:35 +00:00
Renovate Bot
81d5fb60d1
Update dependency @sentry/node to v5.29.0 2020-12-07 11:18:56 +00:00
Daniel Lockyer
12470de19c
Merged 3.39.2 into master
v3.39.2

* tag '3.39.2':
  v3.39.2
  Updated Ghost-Admin to v3.39.2
  Added guard for page.items existing in Mailgun response
2020-12-07 11:17:37 +00:00
Daniel Lockyer
3d61603719
v3.39.2 2020-12-07 11:14:54 +00:00
Daniel Lockyer
cf744daa51
Updated Ghost-Admin to v3.39.2 2020-12-07 11:14:54 +00:00
Kevin Ansfield
2fe94f34ff
Added guard for page.items existing in Mailgun response
no issue

- it's possible to get Mailgun responses where the `items` array doesn't exist so we need to guard against that so we don't error
2020-12-07 11:12:03 +00:00
Kevin Ansfield
84ae8585c6 Added guard for page.items existing in Mailgun response
no issue

- it's possible to get Mailgun responses where the `items` array doesn't exist so we need to guard against that so we don't error
2020-12-07 11:07:25 +00:00
Daniel Lockyer
f351357159
Merged 3.39.1 into master
v3.39.1

* tag '3.39.1':
  v3.39.1
  Updated Ghost-Admin to v3.39.1
2020-12-07 09:23:12 +00:00
Daniel Lockyer
0f6f5d09db
v3.39.1 2020-12-07 09:20:09 +00:00
Daniel Lockyer
801729cd7f
Updated Ghost-Admin to v3.39.1 2020-12-07 09:20:09 +00:00
Renovate Bot
52b0f970e9
Update dependency eslint to v7.15.0 2020-12-07 00:07:07 +00:00
Kevin Ansfield
7c57ffd6a6 Fixed posts without open rate being ordered in reverse
refs https://github.com/TryGhost/Ghost/issues/12420

- adds additional sorting by post creation date so that posts below those with an open rate appear in a more useful order
2020-12-04 13:12:14 +00:00
Thibaut Patel
7038f381b3 🐛 Fixed page preview
issue #12444
2020-12-04 11:35:15 +01:00
Daniel Lockyer
a7e17c6b98
Changed URL protocol in oembed acceptance tests
- `https` was getting caught somewhere with nock and metascraper and
  caused each test case to hang for 3 seconds
- `http` still tests what we want and is instant
2020-12-03 21:46:01 +00:00
Kevin Ansfield
9ff7423b2b
Added email.open_rate order option to posts api (#12439)
refs https://github.com/TryGhost/Ghost/issues/12420

- updated `order` bookshelf plugin's `parseOrderOption()` method to return multiple order-related properties
  - `order` same as before, a key-value object of property-direction
  - `orderRaw` new property that is a raw SQL order string generated from `orderRawQuery()` method in models
  - `eagerLoad` new property that is an array of properties the `eagerLoad` plugin should use to join across
- updated `pagination.fetchAll()` to apply normal order + raw order if both are available and to handle eager loading / joins when `options.eagerLoad` is populated
- updated post model to include details for email relationship and to add `orderRawQuery()` that allows `email.open_rate` to be used as an order option
2020-12-03 20:13:37 +00:00
Daniel Lockyer
47397ca7a6
Updated logging message for webhooks
- removes double space and tidies up the output
2020-12-03 17:50:53 +00:00
Daniel Lockyer
04fb7f8e69
Refactored acceptance tests to use async-await
- refactoring the acceptance tests to use async-await removes all the Promise
  chaining we had, and streamlines the coding styles we have across the code so
  test files are more alike
2020-12-03 08:47:01 +00:00
Renovate Bot
30a98c0185 Update dependency grunt-mocha-cli to v7 2020-12-03 08:46:48 +00:00
Daniel Lockyer
4ef019d88d Deleted v3 regression tests
- at the time of writing, the v3 API === canary API
- we have both v3 + canary regression tests, which are nearly the same
  but there are slight deviations that we keep missing when adding new
  tests
- the canary tests are actually describing functionality of the v3 API
- therefore, we should be ok to delete the v3 regression tests for now
- when v3 is stable, we can copy the canary tests back to v3
2020-12-02 18:27:29 +00:00
Renovate Bot
2d621c9680
Update dependency sanitize-html to v2.2.0 2020-12-02 15:32:57 +00:00
Daniel Lockyer
07354ab1ec
v3.39.0 2020-12-02 15:11:33 +00:00
Daniel Lockyer
40f9bf3769
Updated Ghost-Admin to v3.39.0 2020-12-02 15:11:32 +00:00
Kevin Ansfield
a945ae7c02 Added sentry error capturing to email analytics jobs
no issue

- jobs operate in their own process and so do not share any of the error capturing that is configured in the parent process
2020-12-02 14:57:15 +00:00