Commit Graph

12067 Commits

Author SHA1 Message Date
Daniel Lockyer
622a57fc26
Added missing canary test present in v3 tests
- this test was present in the v3 test suite but not in the canary ones
- given v3 == canary currently, these tests should essentially be the
  same
- copied the test over
2020-12-01 09:35:24 +00:00
Kevin Ansfield
b35eef1b02 Added kill-switch config for email analytics background jobs
no issue

- it's possible background jobs may cause unintended side-effects so it's useful to have a kill-switch to disable them individually to keep sites working
2020-12-01 09:10:59 +00:00
Daniel Lockyer
8f86885132 Fixed typo in v2 oembed API
- canary uses `maxwidth` and it's the correct parameter to use
2020-12-01 08:59:41 +00:00
Kevin Ansfield
5c5bf2d3b0 Fixed oembed tests being network dependent
no issue

- some tests started failing locally because example.com was not resolvable
- mocked dns lookup for all tests so that we're always testing against expected public/private IP address blocks
2020-11-30 21:44:15 +00:00
Scott Cabot
da5586e5aa Fixed user import with no created_at date breaking graph
- users imported from CSV with no created_at date where having their created_at date being stored as an int rather than a datetime.
- this was causing parsing issues with the graph so this commit fixes the formatting
2020-11-30 11:21:22 +00:00
Scott Cabot
1e5d201835 🐛 Fixed flat member chart for GMT-X timezones
closes #12083

- fixes a parsing issue where negative offset values were incorrectly having the + sign added regardless of actual offset for sqlite databases.
- for mysql databases absolute values of offset were taken with sign applied where appropriate to stop issues where both hours and minutes could be negative which would cause both an issue with offsets that could present as -2:30 and by the look of the code also trigger extra padding to result in -2:-030 rather than the expected -2:30
2020-11-30 11:21:22 +00:00
Renovate Bot
99dcf43a45 Update dependency @tryghost/job-manager to v0.3.3 2020-11-30 11:16:25 +00:00
Naz
52bcf6086a Fixed malformatted JSDoc {Object} parameters
no issue

- Removed † symbol from the source code
- Not sure how this symbol got into the source code but it definitely should not be here
2020-11-30 17:18:13 +13:00
Naz
eb52cb06bb Improved threading performance on non-SQLite clients
refs https://github.com/TryGhost/Ghost/pull/12431

- child_process are more expensive to run comparing to worker_threads (ref. https://wanago.io/2019/05/20/node-js-typescript-14-performance-hooks/)
- limiting child process only to sqlite backed instances allows high load servers running with MySQL (most usecases) to continue using worker_threads
2020-11-30 13:27:40 +13:00
Kevin Ansfield
a29ac2691a
Fixed sqlite3 errors when email analytics jobs run (#12431)
no issue

- the 4.2.0 version of `sqlite3` that we're using is not compatible with `worker_threads`
- 5.0.0 should add support this but there are other errors
- 5.0.1 is released but not published (https://github.com/mapbox/node-sqlite3/issues/1386)
2020-11-26 15:12:12 +00:00
Daniel Lockyer
40064a395a Switched frontend i18n requires to go through proxy
- we export i18n from `core/frontend/services/proxy` and this is used in
  the most of the places in the frontend code
- this commit aligns the rest of the code in core/frontend to use the
  proxy too
- unfortunately core/frontend/services/themes/i18n.js loops back to the
  proxy so we have a circular dependency
2020-11-26 14:00:28 +00:00
Kevin Ansfield
dcafebe379 Fixed linting 2020-11-26 13:11:18 +00:00
Kevin Ansfield
717543835c
Added email analytics service (#12393)
no issue

- added `EmailAnalyticsService`
  - `.fetchAll()` grabs and processes all available events
  - `.fetchLatest()` grabs and processes all events since the last seen event timestamp
  - `EventProcessor` passed event objects and updates `email_recipients` or `members` records depending on the event being analytics or list hygiene
    - always returns a `EventProcessingResult` instance so that progress can be tracked and merged across individual events, batches (pages of events), and total runs
    - adds email_id and member_id to the returned result where appropriate so that the stats aggregator can limit processing to data that has changed
    - sets `email_recipients.{delivered_at, opened_at, failed_at}` for analytics events
    - sets `members.subscribed = false` for permanent failure/unsubscribed/complained list hygiene events
  - `StatsAggregator` takes an `EventProcessingResult`-like object containing arrays of email ids and member ids on which to aggregate statistics.
  - jobs for `fetch-latest` and `fetch-all` ready for use with the JobsService
- added `initialiseRecurringJobs()` function to Ghost bootup procedure that schedules the email analytics "fetch latest" job to run every minute
2020-11-26 13:09:38 +00:00
Renovate Bot
c8ec1067c5 Update dependency @tryghost/adapter-manager to v0.2.3 2020-11-26 10:39:44 +00:00
Renovate Bot
d58337b229 Update dependency @tryghost/job-manager to v0.3.2 2020-11-26 10:39:24 +00:00
Kevin Ansfield
0c59b948fa
Added migrations for email analytics (#12387)
no issue

- cleans up unused tables `emails.{meta,stats}`
- adds timestamp columns `email_recipients.{delivered_at,opened_at,failed_at}` that can be used for event timelines and basic stats aggregation
  - indexed because we want to sort by these columns to find the "latest event" when limiting Mailgun events API requests
- adds aggregated stats columns `emails.{delivered_count,opened_count,failed_count}`
- adds a composite index on `email_recipients.[email_id,member_email]` to dramatically speed up `email_recipient` update queries when processing events
  - modifies the db initialisation to support an `'@@INDEXES@@'` key in table schema definition for composite indexes
2020-11-25 17:48:24 +00:00
Fabien 'egg' O'Carroll
cbaf6e5a74
🐛 Fixed backwards compatibility for newsletters (#12422)
closes https://github.com/TryGhost/Ghost/issues/12416

This fixes compatibility for the `send_email_when_published` option for
the Posts API.

The model layer only allows setting the `email_recipient_filter` column
when the `status` is being changed. Because of this we need to withhold
the `status` change until after we have determined the
`email_recipient_filter`.
2020-11-25 13:58:21 +00:00
Daniel Lockyer
d14f174f35 Refactored 'add column' migrations to use helper
- this removes a lot of code duplication and streamlines how we do
  things across the codebase
2020-11-25 13:56:10 +00:00
Daniel Lockyer
5c3439dc86 Refactored 'add table' migrations to use helper
- this removes a lot of code duplication and streamlines how we do
  things across the codebase
2020-11-25 13:56:10 +00:00
Daniel Lockyer
ed2a21bef4 Added JSDoc comments into package-json lib 2020-11-25 11:29:04 +00:00
Daniel Lockyer
90f7cf9e67 Reverted "Refactored remaining function in package-json lib to use async-await"
- this reverts commit d4b15141a0.
- reverting until I can figure out why the tests are broken
2020-11-25 10:57:55 +00:00
Daniel Lockyer
d4b15141a0 Refactored remaining function in package-json lib to use async-await
- this helps simplify the code and gets rid of Promise chaining
- apparently I can't easily use an async function within filter, so I've
  left it for now
2020-11-25 10:44:15 +00:00
Fabien 'egg' O'Carroll
18b87d9734
Included all subscriptions in stripeSubscriptions (#12414)
refs https://github.com/TryGhost/Ghost/issues/12256

We no longer want to filter out cancelled subscriptions, so we are able
to remove the whereIn clause of the relation.

* Fixed paid flag on member
* Fixed content gating for members

Now that the subscriptions for a member include all of them, we must
explicitly check that the member has an active subscription in order to
consider them "paid"
2020-11-25 10:39:07 +00:00
Renovate Bot
200e83c75d Update dependency nconf to v0.11.0 2020-11-25 09:34:09 +00:00
Daniel Lockyer
ac17b6bc35 Refactored package-json lib to use more async-await
- this helps get rid of all the promise chaining and indentation,
  resulting in cleaner code
2020-11-25 09:19:35 +00:00
Daniel Lockyer
4378435e12 Refactored package-json lib into a class
- this helps bring all the code together so we can extract it in the
  future
- turning it into a class also lets us easily inject the i18n instance
  and store it locally
2020-11-25 09:19:35 +00:00
Kevin Ansfield
47e81e3ca1 Fixed error when creating emails when track opens is disabled
no issue

- ensure `email.track_opens` is a boolean rather than `null`
2020-11-25 08:08:15 +00:00
Naz
0e78f2ed63 Improved job queue logging
no issue

- Makes debugging scheduled jobs easier by listing running/scheduled job names
2020-11-25 13:28:13 +13:00
Renovate Bot
dec93a4b72 Update dependency @tryghost/members-ssr to v0.8.6 2020-11-24 10:45:36 +00:00
Renovate Bot
235185f212 Update dependency @tryghost/members-csv to v0.3.3 2020-11-24 08:15:28 +00:00
Renovate Bot
6a9f5b48e9 Update dependency @tryghost/magic-link to v0.6.2 2020-11-24 08:15:08 +00:00
Fabien 'egg' O'Carroll
217f069085
Included cancellation_reason in subscription serialization (#12411)
refs #12403 

This new property needed to be exposed in serialization so that Ghost-Admin
can use it via the API.
2020-11-23 21:19:27 +00:00
Kevin Ansfield
7bbbeeb5b0 Fixed linting 2020-11-23 19:58:34 +00:00
Kevin Ansfield
a8d8598003 Updated mega service to use open tracking from settings
refs https://github.com/TryGhost/Ghost/issues/12390

- switched from enabling open tracking based on `enableDeveloperExperiments` to using the track-opens setting
2020-11-23 18:53:55 +00:00
Kevin Ansfield
ea77f3f349
Added "email_track_opens" setting (#12404)
refs https://github.com/TryGhost/Ghost/issues/12390

- adds "email_track_opens" setting to the `email` group, defaulting to `true`
2020-11-23 18:36:12 +00:00
Kevin Ansfield
4c96aa5c95 Fixed bulk email only having 'bulk-email' tag in certain circumstances
no issue

- the `'bulk-email`' tag was only being added to bulk emails if another more specific tag was set up via config
- we always want the `'bulk-email'` tag to be present for better event filtering
2020-11-23 18:34:17 +00:00
Daniel Lockyer
15650f9dad Merged 3.38.3 into master
v3.38.3

* tag '3.38.3':
  v3.38.3
  Updated Ghost-Admin to v3.38.3
  Fixed sans-serif link style
2020-11-23 17:37:51 +00:00
Fabien 'egg' O'Carroll
b9eb835498
Supported cancellation_reason in @tryghost/members-api (#12406)
refs #12403 

This adds support for passing a cancellation_reason when updating a
subscription to cancel_at_end_of_period. The reason will be stored on
the Subscription metadata, as well as in the cancellation_reason column
in the members_stripe_customers_subscriptions table.

refs https://github.com/TryGhost/Members/pull/221
refs https://github.com/TryGhost/Members/pull/222
2020-11-23 17:34:40 +00:00
Daniel Lockyer
1f0e7daa6f v3.38.3 2020-11-23 17:34:33 +00:00
Daniel Lockyer
ce3dee0dd4 Updated Ghost-Admin to v3.38.3 2020-11-23 17:34:33 +00:00
Fabien 'egg' O'Carroll
47e266e6d3
Added cancellation_reason column to stripe subscriptions (#12405)
refs #12403 

This will allow us to store the reason for each individual subscription
cancellation.
2020-11-23 15:36:14 +00:00
Peter Zimon
03dae728cb Fixed sans-serif link style
no refs.
- fixed regression that links in emails with sans-serif setting wasn't underlined
2020-11-23 15:28:25 +00:00
Peter Zimon
bf9820f97d Fixed sans-serif link style
no refs.
- fixed regression that links in emails with sans-serif setting wasn't underlined
2020-11-23 15:50:02 +01:00
Renovate Bot
3d85625fb1
Update dependency @sentry/node to v5.27.6 2020-11-23 12:45:54 +00:00
Daniel Lockyer
fc9e107254 Revert "Bumped Node version used for Ghost-CLI tests"
- this reverts commit a43f4dc4de
- we can't use Node 12 here because Ghost v1 doesn't support it...
2020-11-23 11:22:23 +00:00
Daniel Lockyer
a43f4dc4de Bumped Node version used for Ghost-CLI tests
- our recommended version is Node 12 so we should use this
- possibly also solves some random CI failures we were seeing with
  sqlite3 not installing?
2020-11-23 11:18:48 +00:00
Renovate Bot
68856821db
Update dependency @sentry/node to v5.27.5 2020-11-23 10:39:17 +00:00
Renovate Bot
48ab8e2e6c Update dependency @tryghost/session-service to v0.1.12 2020-11-23 09:55:05 +00:00
Renovate Bot
7fa0669ce5 Update dependency @tryghost/mw-session-from-token to v0.1.11 2020-11-23 09:54:56 +00:00
Renovate Bot
25bd4253a6 Update dependency @tryghost/adapter-manager to v0.2.2 2020-11-23 09:54:45 +00:00