Commit Graph

8353 Commits

Author SHA1 Message Date
Kukhyeon Heo
7528ec8c3b
🐛 Fixed redirects "to" query params forwarding (#12333)
ref #10898

- The redirects configuration's `to` & `from` URL parameters used to ignore it's query string parameters, which resulted in unexpected behavior
- Current changeset only partially fixes the issue. Now `to` URL's query parameters always take precedence over incoming query parameters and the rest of query parameters are passed through.
2021-01-05 14:11:06 +13:00
Fabien O'Carroll
5af0b5735b 🐛 Added translation for BookshelfRelationsError
refs #12493

This protects against accidental relation errors giving unfriendly
errors messages.
2021-01-04 15:35:43 +00:00
Naz
33bdd2384b 🐛 Fixed incorrect locale loading when changing themes
closes #12271

- When previous active theme did not have locale data for certain language, loading a theme which has such data did not result in correct locale being loaded
- Underlying issue was in settings cache being outdated during theme change related i18n initialization
- Fix focuses on removing settings cache dependency and and rely on most up to date data about currently active theme
- The benefit of this approach is reduced coupling with settings cache
2021-01-04 17:26:22 +13:00
Sumedh
e756af65f2 Removed invalid default 'core' value in settings type
closes #12181
refs https://github.com/TryGhost/Ghost/pull/12265

- The 'core' value is invalid for settings 'type' column
- The 'core' default value for 'type' column in model is also invalid
- Both need to be removed as they are never used and only introduce confusion into the codebase
2021-01-04 15:32:57 +13:00
Daniel Lockyer
ea3ebb4735
Updated Ghost-Admin to v3.40.2 2020-12-16 14:04:43 +00:00
Naz
31a69d18f0 Improved JSDoc in default scheduler
no issue

- Previously used notation was not a valid JSDoc one. With this correction IDEs will provide more accurate information about all furnction parameters
2020-12-16 17:25:05 +13:00
Naz
c1e3788570 Added central error handler to job manager
refs https://github.com/TryGhost/Ghost-Utils/issues/118

- Duplicating error handling across jobs is not best developer experience. Also, having custom error handling logic did not allow for recommended worker script behavior: allowing for unhandled exceptions to bubble up and be managed by parent process
2020-12-14 18:01:41 +13:00
Kevin Ansfield
8aa55feaf8
Added acceptance test for /member/:id/?include=email_recipients (#12477)
refs c1d66f0b01

- fixed base model allowing '@@INDEXES@@' as a permitted attribute/order
- fixed base model automatically setting `@@INDEXES@@` to null on the model when creating
- added `doAuth('members:emails')`
  - creates an `email_batch` record attached to the first email in the fixtures
  - creates an `email_recipients` record for each member
  - runs analytics aggregation so the email and member counts are as expected
- added acceptance test for `/member/:id/?include=email_recipients`
2020-12-11 18:45:35 +00:00
Daniel Lockyer
0c0b5fb1e0
Updated Ghost-Admin to v3.40.1 2020-12-11 11:28:15 +00:00
Daniel Lockyer
31ba8029b0
Updated Ghost-Admin to v3.40.0 2020-12-10 15:49:37 +00:00
Fabien O'Carroll
939a54cc35 🐛 Fixed comped flag for members
no-issue

We now include all subscriptions as part of the member, so we need to
ensure the comped flag is only true if the member has an **active**
complimentary plan
2020-12-10 13:14:07 +00:00
Peter Zimon
c856d712ec
Import email refinements (#12473)
* Updated import email subject

* Updated import email title

* Fixed copy in import email to reflect if there was at least one member added
2020-12-10 12:19:26 +01:00
Thibaut Patel
63dde5a46c
Made the package-json module ready to be exported (#12451)
no issue
2020-12-10 11:37:43 +01:00
Kevin Ansfield
c1d66f0b01
Added email_recipients include option to members API read endpoint (#12471)
refs https://github.com/TryGhost/Ghost-Admin/pull/1796

We want to be able to display an email activity timeline in Ghost-Admin for each member. The quickest way to achieve that right now is to provide access to the `email_recipient` data for the member when fetching, this will allow clients to build up a timeline based on the event timestamps included with each email_recipient/email pair.

- sets up `email_recipients` relationship in `Member` model
- updates members API read endpoint to accept an `email_recipients` include parameter
  - appends `email_recipients.email` to the `withRelated` array when `email_recipients` is included so that we have data available for email subject and html/plaintext for previews
- updates members API output serializer to include the email_recipients object in the output
2020-12-10 10:04:05 +00:00
Kevin Ansfield
9586d1ce56 Fixed sqlite storing member email open rate as floats
no issue

- sqlite will store a float in an integer column due to it's type affinity resulting in long decimal numbers in the UI when we're expecting an integer
- use the `ROUND()` function to ensure we're storing integers in place of floats when performing open rate average calculations
2020-12-10 10:00:25 +00:00
Fabien O'Carroll
32fe260763 Updated canary members controller to use new Importer
no-issue

This completely replaces the old import functionality with the new
importer!
2020-12-09 19:25:57 +00:00
Fabien O'Carroll
cfad396652 Exported MembersCSVImporter from Members service
no-issue

This will allow the controller to be updated to use the new importer
2020-12-09 19:25:57 +00:00
Fabien O'Carroll
ff7773d1cf Added new members importer module
no-issue

This module encapsulates the work around performing imports, it
currently uses the concept of a "Job" which at the moment is not
persisted to the database, however when we want to look at resuming
imports after a server restart, this should give us the flexibility to
do it.
2020-12-09 19:25:57 +00:00
Fabien O'Carroll
c7359a083f Added email template for completion of background jobs
no-issue

This will be used to generate email content used for notifying users
that their import has been completed.
2020-12-09 19:25:57 +00:00
Fabien O'Carroll
73dc19e498 Removed Members CSV import validation endpoint
no-issue

The new import flow does not use prevalidation of the CSV file, so we
have no use for this anymore.
2020-12-09 19:25:57 +00:00
Fabien O'Carroll
83c5270217 Removed support for batched CSV importer
no-issue

We are rewriting the Members CSV importer to use background jobs, the
batched importer will no longer be used locally.
2020-12-09 19:25:57 +00:00
Kevin Ansfield
baf635e27b Updated email stats aggregator for new member count columns
refs https://github.com/TryGhost/Ghost/issues/12461

- added two default aggregations for overall email count and opened email count
- when number of tracked emails is sufficient add the open rate aggregation to the update query
2020-12-09 13:14:39 +00:00
Kevin Ansfield
b1aafd715d
Added members.email_{count,opened_count} column migrations (#12470)
refs https://github.com/TryGhost/Ghost/issues/12461

- adds `members.email_count` and `members.email_opened_count` columns to contain cached counts for faster queries when outputting member data via API
- adds migration to populate cached counts with existing data
  - tested locally on ~50k members which took ~4sec on mysql
- updates members output serializer to include the new fields in API output
2020-12-09 12:21:56 +00:00
Thibaut Patel
c25344d414
Refactored core/server/lib/image for Dependency Injection (#12463)
no issue
2020-12-09 13:19:22 +01:00
Kevin Ansfield
748a8e0d0f
Added email_open_rate order option to members API (#12462)
refs https://github.com/TryGhost/Ghost/issues/12421

- add `orderRawQuery` function to members model so that we can ensure members with an open rate are ordered before members without an open rate no matter the order direction chosen
- added `email_open_rate` to members in the test fixtures to allow testing of order
2020-12-08 12:43:48 +00:00
Kevin Ansfield
567eb6325f
Added members.email_open_rate aggregation to email analytics (#12458)
refs https://github.com/TryGhost/Ghost/issues/12421
requires https://github.com/TryGhost/Ghost/pull/12457

- updates stats aggregator to calculate and store an open rate for each member
  - uses two queries because I couldn't find a reasonable approach to perform the update in a single query as per the email aggregation
  - benchmarked locally at <1sec/1000members
  - will not store an open rate unless the number of tracked emails sent to a member is above a certain threshold (defaults to 5) to avoid new members being heavily weighted
- fixes typo in EmailAnalytics that was stopping member stats from being aggregated
2020-12-08 12:43:10 +00:00
Kevin Ansfield
9fd6f30fd7
Added migration for members.email_open_rate column (#12457)
refs https://github.com/TryGhost/Ghost/issues/12421

- nullable so we can distinguish between members that have and haven't received any trackable emails
- indexed because we'll be using this column for sorting
2020-12-08 12:20:17 +00:00
Daniel Lockyer
99632eab43
Merged 3.39.3 into master
v3.39.3

* tag '3.39.3':
  v3.39.3
  Updated Ghost-Admin to v3.39.3
2020-12-08 09:42:01 +00:00
Daniel Lockyer
894d18c641
Updated Ghost-Admin to v3.39.3 2020-12-08 09:37:00 +00:00
Daniel Lockyer
12470de19c
Merged 3.39.2 into master
v3.39.2

* tag '3.39.2':
  v3.39.2
  Updated Ghost-Admin to v3.39.2
  Added guard for page.items existing in Mailgun response
2020-12-07 11:17:37 +00:00
Daniel Lockyer
cf744daa51
Updated Ghost-Admin to v3.39.2 2020-12-07 11:14:54 +00:00
Kevin Ansfield
2fe94f34ff
Added guard for page.items existing in Mailgun response
no issue

- it's possible to get Mailgun responses where the `items` array doesn't exist so we need to guard against that so we don't error
2020-12-07 11:12:03 +00:00
Kevin Ansfield
84ae8585c6 Added guard for page.items existing in Mailgun response
no issue

- it's possible to get Mailgun responses where the `items` array doesn't exist so we need to guard against that so we don't error
2020-12-07 11:07:25 +00:00
Daniel Lockyer
f351357159
Merged 3.39.1 into master
v3.39.1

* tag '3.39.1':
  v3.39.1
  Updated Ghost-Admin to v3.39.1
2020-12-07 09:23:12 +00:00
Daniel Lockyer
801729cd7f
Updated Ghost-Admin to v3.39.1 2020-12-07 09:20:09 +00:00
Kevin Ansfield
7c57ffd6a6 Fixed posts without open rate being ordered in reverse
refs https://github.com/TryGhost/Ghost/issues/12420

- adds additional sorting by post creation date so that posts below those with an open rate appear in a more useful order
2020-12-04 13:12:14 +00:00
Thibaut Patel
7038f381b3 🐛 Fixed page preview
issue #12444
2020-12-04 11:35:15 +01:00
Kevin Ansfield
9ff7423b2b
Added email.open_rate order option to posts api (#12439)
refs https://github.com/TryGhost/Ghost/issues/12420

- updated `order` bookshelf plugin's `parseOrderOption()` method to return multiple order-related properties
  - `order` same as before, a key-value object of property-direction
  - `orderRaw` new property that is a raw SQL order string generated from `orderRawQuery()` method in models
  - `eagerLoad` new property that is an array of properties the `eagerLoad` plugin should use to join across
- updated `pagination.fetchAll()` to apply normal order + raw order if both are available and to handle eager loading / joins when `options.eagerLoad` is populated
- updated post model to include details for email relationship and to add `orderRawQuery()` that allows `email.open_rate` to be used as an order option
2020-12-03 20:13:37 +00:00
Daniel Lockyer
47397ca7a6
Updated logging message for webhooks
- removes double space and tidies up the output
2020-12-03 17:50:53 +00:00
Daniel Lockyer
40f9bf3769
Updated Ghost-Admin to v3.39.0 2020-12-02 15:11:32 +00:00
Kevin Ansfield
a945ae7c02 Added sentry error capturing to email analytics jobs
no issue

- jobs operate in their own process and so do not share any of the error capturing that is configured in the parent process
2020-12-02 14:57:15 +00:00
Kevin Ansfield
c40a8e978d Fixed incorrect knex syntax
refs 166d072cd4

- accidentally committed an earlier attempt 🤦🏻‍♂️ updated with the valid syntax
2020-12-02 14:53:10 +00:00
Kevin Ansfield
166d072cd4 Fixed email analytics job not being registered when creating an email
no issue

- job registration was checking for submitted emails in it's email count but the job registration method is called as soon as an email is created meaning the email has a status of 'pending' which prevented the analytics job from being started until a second email was sent
2020-12-02 14:47:18 +00:00
Kevin Ansfield
f802128cfc
Added emailAnalytics config feature flag (#12443)
no issue

- email analytics may be desirable to fully switch off in certain circumstances, when that happens we want to prevent related background jobs from running and expose the feature flag via the config endpoint in the Admin API so that clients can adjust accordingly
2020-12-02 13:22:12 +00:00
Kevin Ansfield
2951eb9eaf Added missing moment require
refs 92ef83c61a
2020-12-02 12:15:27 +00:00
Kevin Ansfield
92ef83c61a Adjusted email jobs registration to check for emails newer than 30 days
no issue

- if emails are older than 30 days we wouldn't be able to fetch any analytics for them and if a site used emails in the past but is no longer using them it doesn't make sense to keep potentially expensive background worker threads spinning up
2020-12-02 12:14:22 +00:00
Daniel Lockyer
947603d0e3
Added original error to thrown error in mega code
- the original error is not propagated up with the error we throw, so it
  can sometimes be difficult to see what's going on down the line
2020-12-02 09:04:20 +00:00
Kevin Ansfield
d675278b0b
Prevented scheduling of recurring analytics jobs when not using emails (#12441)
no issue

- recurring jobs spin up worker threads which can be quite CPU intensive even when not performing much processing, this can be problematic in environments where there are many Ghost instances running
- updated the email job scheduling to be skipped on bootup when there are no emails in the database and to be started when the first email is created as long as we're not in testing env
- increase analytics job schedule from every 2 minutes to every 5 minutes to help spread the load further across instances
2020-12-02 08:17:44 +00:00
Kevin Ansfield
249fd4f06a Fixed email count check in email-analytics service
no issue

- raw knex `.count()` does not return a straight number, we need to handle an array of rowDataPacket objects
2020-12-01 15:02:04 +00:00
Kevin Ansfield
641be8fec3 Adjusted email analytics job scheduling to avoid API spikes
no issue

- typically cron/later schedules will schedule for :00 on the minute which would create API spikes with every members-email-using Ghost site hitting the API at the same time
- adjusted the scheduling to use cron syntax with job runs every 2 minutes on 1,3,5... or 2,4,6... and a random seconds value to smooth usage across sites
2020-12-01 11:27:34 +00:00