Commit Graph

11703 Commits

Author SHA1 Message Date
Renovate Bot
98be0a9a9f
Update dependency @sentry/node to v5.22.0 2020-08-27 10:54:58 +00:00
Nazar Gargol
e8248cdc9e Refactored members imporeter in preparation for jobs
no issue

- This refactor extracts labels related code into a separate module for easier reuse by the "job-aware" batched importer
2020-08-27 21:47:00 +12:00
Nazar Gargol
bcca59ffbe Fixed regression tests for members importer
refs 262d33b73c

- Added missing test changes from referenced commit
2020-08-27 19:28:18 +12:00
Nazar Gargol
262d33b73c Improved import_label creation logic
no issue

- When an import was done and there were no "global labels" present Ghost created generic `import-[data]` label which later helped to find a specific batch of imported data
- It did not make sense to create such generic label when user provided their own unique label
- The rules that work now are:
1. When there is no global provided Ghost generates on and removes it in case there are no imported records
2. When there is a unique new global label provided no new label is generated, but the label stays even if there are no imported records
2020-08-27 19:07:27 +12:00
Kevin Ansfield
f30d3cd2c2 Switched to bookshelf count in getTotalMembers()
no issue

- tested performance between knex raw, knex `count()` and bookshelf `count()` and found no difference over 1000 iterations of each (each ~19,500ms +- 500ms for 104k members locally)
- switched to using bookshelf as the code is the simplest
2020-08-27 01:52:36 +01:00
Kevin Ansfield
a995e9cc89 Allowed custom and search queries to be used with .findAll()
no issue

- extract filtering of an collection into a separate function
- use extracted function in `findAll()` so that it's query behaviour matches `findPage()`
2020-08-27 01:09:07 +01:00
Kevin Ansfield
5e64f113d5 Skipped separate count query in .findPage() for limit="all" requests
no issue

- for large result sets or complex queries the count query itself can be quite time consuming
- when `limit: 'all'` is passed as an option there's no need to perform a separate count query because we can determine the pagination data from the final result set
- skipped count query when `limit: 'all'` option is present
- re-ordered comments to be closer to the code they reference (ie, why we have our own count query instead of Bookshelf's `.count()`
2020-08-27 01:09:07 +01:00
Daniel Lockyer
e3eed7c02d Revert "Update dependency oembed-parser to v1.4.1"
- this reverts commit 054689ee4e.
- this update cannot be merged until Ghost's minimum Node versions are
  bumped
- somehow it snook through our CI checks and got merged by Renovate
2020-08-26 17:33:35 +01:00
Renovate Bot
054689ee4e Update dependency oembed-parser to v1.4.1 2020-08-26 16:27:57 +00:00
Renovate Bot
9e8d3828b6
Update dependency sanitize-html to v1.27.4 2020-08-26 14:59:35 +00:00
Nazar Gargol
9fe5007929 Removed date handling validation in members importer logic
no issue

- This is handled on input sanitization layer with date
format check in JSON schema validation, so there's no need to do this
check again in the importer.
2020-08-26 17:25:24 +12:00
Nazar Gargol
544382b651 Updated boolean values validations for members uploads
no issue

- When the value is part of `enum` the `null` value is no longer
recognized by `null` type in JSON schema validator.
2020-08-26 17:23:36 +12:00
Nazar Gargol
95044e3ba0 Added error handling for failed member imports
no issue

- When bulk insert fails there is no transactional logic to revert
related records form being inserted. Also, previously there were no
attempts to "retry" the insert.
- To avoid complex retry logic, an iterative one-by-one insert retry
approach was taken. If this becomes a bottleneck in the future, the
retry algorithm could be improved.
- To avoid a lot of code duplication refactored model's `bulkAdd` & `bulkDestroy`
methods to use 'bulk-operations' module.
- Updated error handling and logging for bulk delete operations. It's very
unlikely for error to happen here,  but still need to make sure there is
a proper logging in place to trace back the failure.
- Added debug logs. This should improve debugging experience and
performance measurements.
- Added handling for unrecognized errors. Handling inspired by current unrecognized
error handling by ghost importer -10e5d5f3d4/core/server/data/importer/importers/data/base.js (L148-L154)
2020-08-26 17:11:35 +12:00
Renovate Bot
9e991fb47c
Update dependency mocha to v8.1.2 2020-08-25 19:51:28 +00:00
Renovate Bot
c50b677a50 Update dependency grunt to v1.3.0 2020-08-25 13:56:18 +01:00
Daniel Lockyer
10e5d5f3d4 v3.31.2 2020-08-25 12:57:57 +01:00
Daniel Lockyer
d9700ba9bc Updated Ghost-Admin to v3.31.2 2020-08-25 12:57:57 +01:00
Nazar Gargol
45bca2b104 Fixed linting for await statements
refs 9d65de4b10
2020-08-25 19:30:02 +12:00
Nazar Gargol
9d65de4b10 Removed created import labels when no records were imported
no issue

- When no members are succesfully imported through CSV import process the import label should not be created. Otherwise after multiple failed attempts to import there are orphaned labels in the system
2020-08-25 19:23:05 +12:00
Kevin Ansfield
e07e0a17c6 🐛 Fixed bookmark card display in member emails when using Gmail+Chrome
no issue

- updates `@tryghost/kg-default-cards` which contains two fixes
  - removes email-specific output being added to post html (had no visual impact due to use of conditional comments but keeps rendered html smaller+cleaner)
  - adds a background-url style to the thumbnail container to give two options for styling
- updates member email template styling to hide the `<img>` element in bookmark cards and use a background image instead to get consistent rendering across email clients
2020-08-24 18:21:09 +01:00
Daniel Lockyer
8b9c161ff9 v3.31.1 2020-08-24 16:29:08 +01:00
Daniel Lockyer
24c5c32fa8 Updated Ghost-Admin to v3.31.1 2020-08-24 16:29:08 +01:00
Renovate Bot
fd2a49a366 Update dependency jwks-rsa to v1.9.0 2020-08-24 15:29:07 +01:00
Renovate Bot
48144df5a8
Update dependency @sentry/node to v5.21.4 2020-08-24 12:25:00 +00:00
Daniel Lockyer
f6a1c330c9 v3.31.0 2020-08-24 11:33:22 +01:00
Daniel Lockyer
50a10a4903 Updated Ghost-Admin to v3.31.0 2020-08-24 11:33:22 +01:00
Peter Zimon
75f5ab637d Added minor member newsletter customisations via config
no issue
- adds a `members:emailTemplate` config object
  - `showSiteHeader` - defaults to `true`, shows the site title and icon in member emails
  - `showPoweredBy` - defaults to `false`, adds a "Publish with Ghost" button to member email footer
- updates member newsletter email template with hideable site header and "powered by" badge
2020-08-24 10:44:22 +01:00
Rish
297b63edce 🐛 Fixed response for members API update endpoint
closes https://github.com/TryGhost/members.js/issues/87

- The `update` method in members-api package was edited to return Model object instead of JSON directly [here](a28bcc5b2a)
- This caused the update member API on member endpoint to return partial response only as most properties couldn't be fetched
- Fix updates the middleware to correctly call `toJSON` before formatting response
2020-08-24 13:38:21 +05:30
Nazar Gargol
91d1417df2 Fixed email newsletter error when post has no content
closes #12158
2020-08-24 19:46:46 +12:00
Nazar Gargol
b8197023b3 🐛 Fixed ignored created_at and subscribed fields in Members API
closes #12156

- When adding new member through `POST /members` API or importing members with CSV importer `POST /members/upload` API created_at and subscribed were ignored
- Similar problem but only with `subscribed` field was present in `PUT /members/:id` API
- The regression was introduced with a bump of @tryghost/members-api to 0.26.0, specifically this change in upstream - a28bcc5b2a (diff-3daeef67d07a2a0f94c89a86cafcede9R44)
- Bumped @tryghost/members-api package to 0.28.2 fixing the underlying issue - 7b5f2e3cb7
2020-08-24 19:35:20 +12:00
Renovate Bot
679616026f
Update dependency @tryghost/members-api to v0.28.2 2020-08-24 06:33:55 +00:00
Kevin Ansfield
cbe2817c72 🐛 Fixed bookmark card display in member emails when using Outlook
closes https://github.com/TryGhost/Ghost/issues/12148

- updates `@tryghost/kg-default-cards`, new version includes Outlook-specific bookmark card template and styling
2020-08-22 11:13:54 +01:00
Renovate Bot
2b635ab318 Update dependency @sentry/node to v5.21.3 2020-08-21 22:08:16 +00:00
Daniel Lockyer
e56921eeb7 Fixed merging of 3.30.2 into master
v3.30.2

* tag '3.30.2':
  v3.30.2
  Updated Ghost-Admin to v3.30.2
2020-08-21 12:20:51 +01:00
Daniel Lockyer
0ab190e83c Merged 3.30.2 into master
* 3.30.2:
  Bumped @tryghost/members-api to 0.28.1 in lockfile
  Bumbed @tryghost/members-api to 0.28.1
  🐛 Fixed unable to delete member when stripe is connected
2020-08-21 12:06:01 +01:00
Daniel Lockyer
d975714138 v3.30.2 2020-08-21 12:03:37 +01:00
Daniel Lockyer
e51ab7707d Updated Ghost-Admin to v3.30.2 2020-08-21 12:03:37 +01:00
Rish
7177da6a4e Bumped @tryghost/members-api to 0.28.1 in lockfile
refs https://github.com/TryGhost/Ghost/issues/12150

- Adds the updated lockfile for previous commit which bumps members-api to 0.28.1
2020-08-21 12:01:31 +01:00
Rish
acd34c9bcb Bumbed @tryghost/members-api to 0.28.1
refs https://github.com/TryGhost/Ghost/issues/12150

- Fixes incorrect stripe cancellation method for member `destroy` call
2020-08-21 11:57:31 +01:00
Rish
4657941095 Bumped @tryghost/members-api to 0.28.1 in lockfile
refs https://github.com/TryGhost/Ghost/issues/12150

- Adds the updated lockfile for previous commit which bumps members-api to 0.28.1
2020-08-21 16:23:37 +05:30
Rish
bf84d42966 Bumbed @tryghost/members-api to 0.28.1
refs https://github.com/TryGhost/Ghost/issues/12150

- Fixes incorrect stripe cancellation method for member `destroy` call
2020-08-21 16:18:01 +05:30
Rish
87e8298ca7 🐛 Fixed unable to delete member when stripe is connected
closes https://github.com/TryGhost/Ghost/issues/12150

- The `destroy` method on members-api method was called incorrectly with not enough params
- It expects both `data` and `options` passed down [here](https://github.com/TryGhost/Members/blob/master/packages/members-api/lib/users.js#L14)
- Missing `options` in method call throws error as we read `cancelStripeSubscriptions` property on options if stripe is connected
- Fix passes both data and options to the destroy method
2020-08-21 11:13:07 +01:00
Rishabh Garg
c7f3837c54
Added admin endpoint for editing member subscription (#12145)
refs #12127

- Adds new `editSubscription` endpoint for members admin API which allows updating individual subscription for a member - `PUT /members/:id/subscriptions/:subscription_id/`
- `editSubscription` has same permissions as member's `edit` endpoint
- Currently allows toggling of cancellation at period end for an active subscription
2020-08-20 17:28:11 +05:30
Renovate Bot
4fff93f9e0
Update dependency @tryghost/members-api to v0.28.0 2020-08-20 09:14:32 +00:00
Daniel Lockyer
441d345759 v3.30.1 2020-08-20 09:28:09 +01:00
Daniel Lockyer
04f42732c8 Updated Ghost-Admin to v3.30.1 2020-08-20 09:28:09 +01:00
Nazar Gargol
5670d99f46 Improved error handling for batch deleted records
no issue

- Similar handling to one introduced in 8418c829de
- Having granular tracking for failed to remove id's would make it possible to return more specific errors to the client
2020-08-20 19:11:41 +12:00
Nazar Gargol
8418c829de Improved error handling for batch inserted records
no issue

- Similar to 3a594ce22e this adds error handling to batch operations done outside models
2020-08-20 19:01:17 +12:00
Nazar Gargol
f8adb4a672 Refactored bulk insert/delete operations into separate module
no issue

- Moved bulk db operations outside of importer module to create clearer separation of responsibilities
2020-08-20 18:42:38 +12:00
naz
3a594ce22e
Improved error handling for batch inserted member records (#12146)
no issue

- When batch insert fails handling should be more granular and aim to retry and insert as many records from the batch as possible.  
- Added retry logic for failed member's batch inserts. It's a sequential insert for each record in the batch. This implementation was chosen to keep it as simple as possible
- Added filtering of "toCreate" records when member fails to insert. We should not try inserting related members_labels/members_stripe_customers/members_stripe_customer_subscriptions records because they would definitely fail insertion without associated member record
2020-08-20 17:41:47 +12:00