Commit Graph

11959 Commits

Author SHA1 Message Date
Naz
eb52cb06bb Improved threading performance on non-SQLite clients
refs https://github.com/TryGhost/Ghost/pull/12431

- child_process are more expensive to run comparing to worker_threads (ref. https://wanago.io/2019/05/20/node-js-typescript-14-performance-hooks/)
- limiting child process only to sqlite backed instances allows high load servers running with MySQL (most usecases) to continue using worker_threads
2020-11-30 13:27:40 +13:00
Kevin Ansfield
a29ac2691a
Fixed sqlite3 errors when email analytics jobs run (#12431)
no issue

- the 4.2.0 version of `sqlite3` that we're using is not compatible with `worker_threads`
- 5.0.0 should add support this but there are other errors
- 5.0.1 is released but not published (https://github.com/mapbox/node-sqlite3/issues/1386)
2020-11-26 15:12:12 +00:00
Daniel Lockyer
40064a395a Switched frontend i18n requires to go through proxy
- we export i18n from `core/frontend/services/proxy` and this is used in
  the most of the places in the frontend code
- this commit aligns the rest of the code in core/frontend to use the
  proxy too
- unfortunately core/frontend/services/themes/i18n.js loops back to the
  proxy so we have a circular dependency
2020-11-26 14:00:28 +00:00
Kevin Ansfield
dcafebe379 Fixed linting 2020-11-26 13:11:18 +00:00
Kevin Ansfield
717543835c
Added email analytics service (#12393)
no issue

- added `EmailAnalyticsService`
  - `.fetchAll()` grabs and processes all available events
  - `.fetchLatest()` grabs and processes all events since the last seen event timestamp
  - `EventProcessor` passed event objects and updates `email_recipients` or `members` records depending on the event being analytics or list hygiene
    - always returns a `EventProcessingResult` instance so that progress can be tracked and merged across individual events, batches (pages of events), and total runs
    - adds email_id and member_id to the returned result where appropriate so that the stats aggregator can limit processing to data that has changed
    - sets `email_recipients.{delivered_at, opened_at, failed_at}` for analytics events
    - sets `members.subscribed = false` for permanent failure/unsubscribed/complained list hygiene events
  - `StatsAggregator` takes an `EventProcessingResult`-like object containing arrays of email ids and member ids on which to aggregate statistics.
  - jobs for `fetch-latest` and `fetch-all` ready for use with the JobsService
- added `initialiseRecurringJobs()` function to Ghost bootup procedure that schedules the email analytics "fetch latest" job to run every minute
2020-11-26 13:09:38 +00:00
Renovate Bot
c8ec1067c5 Update dependency @tryghost/adapter-manager to v0.2.3 2020-11-26 10:39:44 +00:00
Renovate Bot
d58337b229 Update dependency @tryghost/job-manager to v0.3.2 2020-11-26 10:39:24 +00:00
Kevin Ansfield
0c59b948fa
Added migrations for email analytics (#12387)
no issue

- cleans up unused tables `emails.{meta,stats}`
- adds timestamp columns `email_recipients.{delivered_at,opened_at,failed_at}` that can be used for event timelines and basic stats aggregation
  - indexed because we want to sort by these columns to find the "latest event" when limiting Mailgun events API requests
- adds aggregated stats columns `emails.{delivered_count,opened_count,failed_count}`
- adds a composite index on `email_recipients.[email_id,member_email]` to dramatically speed up `email_recipient` update queries when processing events
  - modifies the db initialisation to support an `'@@INDEXES@@'` key in table schema definition for composite indexes
2020-11-25 17:48:24 +00:00
Fabien 'egg' O'Carroll
cbaf6e5a74
🐛 Fixed backwards compatibility for newsletters (#12422)
closes https://github.com/TryGhost/Ghost/issues/12416

This fixes compatibility for the `send_email_when_published` option for
the Posts API.

The model layer only allows setting the `email_recipient_filter` column
when the `status` is being changed. Because of this we need to withhold
the `status` change until after we have determined the
`email_recipient_filter`.
2020-11-25 13:58:21 +00:00
Daniel Lockyer
d14f174f35 Refactored 'add column' migrations to use helper
- this removes a lot of code duplication and streamlines how we do
  things across the codebase
2020-11-25 13:56:10 +00:00
Daniel Lockyer
5c3439dc86 Refactored 'add table' migrations to use helper
- this removes a lot of code duplication and streamlines how we do
  things across the codebase
2020-11-25 13:56:10 +00:00
Daniel Lockyer
ed2a21bef4 Added JSDoc comments into package-json lib 2020-11-25 11:29:04 +00:00
Daniel Lockyer
90f7cf9e67 Reverted "Refactored remaining function in package-json lib to use async-await"
- this reverts commit d4b15141a0.
- reverting until I can figure out why the tests are broken
2020-11-25 10:57:55 +00:00
Daniel Lockyer
d4b15141a0 Refactored remaining function in package-json lib to use async-await
- this helps simplify the code and gets rid of Promise chaining
- apparently I can't easily use an async function within filter, so I've
  left it for now
2020-11-25 10:44:15 +00:00
Fabien 'egg' O'Carroll
18b87d9734
Included all subscriptions in stripeSubscriptions (#12414)
refs https://github.com/TryGhost/Ghost/issues/12256

We no longer want to filter out cancelled subscriptions, so we are able
to remove the whereIn clause of the relation.

* Fixed paid flag on member
* Fixed content gating for members

Now that the subscriptions for a member include all of them, we must
explicitly check that the member has an active subscription in order to
consider them "paid"
2020-11-25 10:39:07 +00:00
Renovate Bot
200e83c75d Update dependency nconf to v0.11.0 2020-11-25 09:34:09 +00:00
Daniel Lockyer
ac17b6bc35 Refactored package-json lib to use more async-await
- this helps get rid of all the promise chaining and indentation,
  resulting in cleaner code
2020-11-25 09:19:35 +00:00
Daniel Lockyer
4378435e12 Refactored package-json lib into a class
- this helps bring all the code together so we can extract it in the
  future
- turning it into a class also lets us easily inject the i18n instance
  and store it locally
2020-11-25 09:19:35 +00:00
Kevin Ansfield
47e81e3ca1 Fixed error when creating emails when track opens is disabled
no issue

- ensure `email.track_opens` is a boolean rather than `null`
2020-11-25 08:08:15 +00:00
Naz
0e78f2ed63 Improved job queue logging
no issue

- Makes debugging scheduled jobs easier by listing running/scheduled job names
2020-11-25 13:28:13 +13:00
Renovate Bot
dec93a4b72 Update dependency @tryghost/members-ssr to v0.8.6 2020-11-24 10:45:36 +00:00
Renovate Bot
235185f212 Update dependency @tryghost/members-csv to v0.3.3 2020-11-24 08:15:28 +00:00
Renovate Bot
6a9f5b48e9 Update dependency @tryghost/magic-link to v0.6.2 2020-11-24 08:15:08 +00:00
Fabien 'egg' O'Carroll
217f069085
Included cancellation_reason in subscription serialization (#12411)
refs #12403 

This new property needed to be exposed in serialization so that Ghost-Admin
can use it via the API.
2020-11-23 21:19:27 +00:00
Kevin Ansfield
7bbbeeb5b0 Fixed linting 2020-11-23 19:58:34 +00:00
Kevin Ansfield
a8d8598003 Updated mega service to use open tracking from settings
refs https://github.com/TryGhost/Ghost/issues/12390

- switched from enabling open tracking based on `enableDeveloperExperiments` to using the track-opens setting
2020-11-23 18:53:55 +00:00
Kevin Ansfield
ea77f3f349
Added "email_track_opens" setting (#12404)
refs https://github.com/TryGhost/Ghost/issues/12390

- adds "email_track_opens" setting to the `email` group, defaulting to `true`
2020-11-23 18:36:12 +00:00
Kevin Ansfield
4c96aa5c95 Fixed bulk email only having 'bulk-email' tag in certain circumstances
no issue

- the `'bulk-email`' tag was only being added to bulk emails if another more specific tag was set up via config
- we always want the `'bulk-email'` tag to be present for better event filtering
2020-11-23 18:34:17 +00:00
Daniel Lockyer
15650f9dad Merged 3.38.3 into master
v3.38.3

* tag '3.38.3':
  v3.38.3
  Updated Ghost-Admin to v3.38.3
  Fixed sans-serif link style
2020-11-23 17:37:51 +00:00
Fabien 'egg' O'Carroll
b9eb835498
Supported cancellation_reason in @tryghost/members-api (#12406)
refs #12403 

This adds support for passing a cancellation_reason when updating a
subscription to cancel_at_end_of_period. The reason will be stored on
the Subscription metadata, as well as in the cancellation_reason column
in the members_stripe_customers_subscriptions table.

refs https://github.com/TryGhost/Members/pull/221
refs https://github.com/TryGhost/Members/pull/222
2020-11-23 17:34:40 +00:00
Daniel Lockyer
1f0e7daa6f v3.38.3 2020-11-23 17:34:33 +00:00
Daniel Lockyer
ce3dee0dd4 Updated Ghost-Admin to v3.38.3 2020-11-23 17:34:33 +00:00
Fabien 'egg' O'Carroll
47e266e6d3
Added cancellation_reason column to stripe subscriptions (#12405)
refs #12403 

This will allow us to store the reason for each individual subscription
cancellation.
2020-11-23 15:36:14 +00:00
Peter Zimon
03dae728cb Fixed sans-serif link style
no refs.
- fixed regression that links in emails with sans-serif setting wasn't underlined
2020-11-23 15:28:25 +00:00
Peter Zimon
bf9820f97d Fixed sans-serif link style
no refs.
- fixed regression that links in emails with sans-serif setting wasn't underlined
2020-11-23 15:50:02 +01:00
Renovate Bot
3d85625fb1
Update dependency @sentry/node to v5.27.6 2020-11-23 12:45:54 +00:00
Daniel Lockyer
fc9e107254 Revert "Bumped Node version used for Ghost-CLI tests"
- this reverts commit a43f4dc4de
- we can't use Node 12 here because Ghost v1 doesn't support it...
2020-11-23 11:22:23 +00:00
Daniel Lockyer
a43f4dc4de Bumped Node version used for Ghost-CLI tests
- our recommended version is Node 12 so we should use this
- possibly also solves some random CI failures we were seeing with
  sqlite3 not installing?
2020-11-23 11:18:48 +00:00
Renovate Bot
68856821db
Update dependency @sentry/node to v5.27.5 2020-11-23 10:39:17 +00:00
Renovate Bot
48ab8e2e6c Update dependency @tryghost/session-service to v0.1.12 2020-11-23 09:55:05 +00:00
Renovate Bot
7fa0669ce5 Update dependency @tryghost/mw-session-from-token to v0.1.11 2020-11-23 09:54:56 +00:00
Renovate Bot
25bd4253a6 Update dependency @tryghost/adapter-manager to v0.2.2 2020-11-23 09:54:45 +00:00
Fabien 'egg' O'Carroll
f8b617af64
Supported redirecting externally after signup (#12391)
refs #12391

Adds support for redirecting to external URL's after successful signup for members.
2020-11-23 09:36:45 +00:00
Naz
5c4e884070 Added support for graceful job shutdown and worker threads
refs #12402

- With bumped version of job-manager it offloads job procesing into separate worker thread. Having jobs run out of main Ghost process even loop allows for safe job execution, which does not block Ghost from serving requests or performing other functions without a delay
- Added experimental data access to 'testmode' jobs. This should serve as an illustration of how to access data from the job layer
2020-11-23 17:17:49 +13:00
Renovate Bot
73911f7ba4
Update dependency eslint to v7.14.0 2020-11-23 00:05:58 +00:00
Naz
6ba6e9c307 Removed unused authors input serializer
refs https://github.com/TryGhost/Ghost/issues/10354

- The serializer wasn't hooked up properly during the implementation (1a4497fc9a). It is not possible to hook it back in now as that would introduce breaking change to now stable v2 API.
- v3 (canary) has is properly hooked in, so there should not be a problem going forward
2020-11-23 11:35:02 +13:00
Thibaut Patel
bda4e0da70 Fixed Personal Tokens migration file name
no ref
2020-11-20 10:35:31 +01:00
Thibaut Patel
5fefa9fe96 Disabled Personal Tokens of inactive/locked users
no issue
2020-11-20 09:53:18 +01:00
Thibaut Patel
d48febbbd8 Added Personal Token removal when a user is deleted
no issue
2020-11-20 09:53:18 +01:00
Thibaut Patel
4edccfd2f7 Added the Admin API and tests for Personal Tokens.
no issue
2020-11-20 09:53:18 +01:00