Commit Graph

8268 Commits

Author SHA1 Message Date
Rish
87e8298ca7 🐛 Fixed unable to delete member when stripe is connected
closes https://github.com/TryGhost/Ghost/issues/12150

- The `destroy` method on members-api method was called incorrectly with not enough params
- It expects both `data` and `options` passed down [here](https://github.com/TryGhost/Members/blob/master/packages/members-api/lib/users.js#L14)
- Missing `options` in method call throws error as we read `cancelStripeSubscriptions` property on options if stripe is connected
- Fix passes both data and options to the destroy method
2020-08-21 11:13:07 +01:00
Rishabh Garg
c7f3837c54
Added admin endpoint for editing member subscription (#12145)
refs #12127

- Adds new `editSubscription` endpoint for members admin API which allows updating individual subscription for a member - `PUT /members/:id/subscriptions/:subscription_id/`
- `editSubscription` has same permissions as member's `edit` endpoint
- Currently allows toggling of cancellation at period end for an active subscription
2020-08-20 17:28:11 +05:30
Daniel Lockyer
04f42732c8 Updated Ghost-Admin to v3.30.1 2020-08-20 09:28:09 +01:00
Nazar Gargol
5670d99f46 Improved error handling for batch deleted records
no issue

- Similar handling to one introduced in 8418c829de
- Having granular tracking for failed to remove id's would make it possible to return more specific errors to the client
2020-08-20 19:11:41 +12:00
Nazar Gargol
8418c829de Improved error handling for batch inserted records
no issue

- Similar to 3a594ce22e this adds error handling to batch operations done outside models
2020-08-20 19:01:17 +12:00
Nazar Gargol
f8adb4a672 Refactored bulk insert/delete operations into separate module
no issue

- Moved bulk db operations outside of importer module to create clearer separation of responsibilities
2020-08-20 18:42:38 +12:00
naz
3a594ce22e
Improved error handling for batch inserted member records (#12146)
no issue

- When batch insert fails handling should be more granular and aim to retry and insert as many records from the batch as possible.  
- Added retry logic for failed member's batch inserts. It's a sequential insert for each record in the batch. This implementation was chosen to keep it as simple as possible
- Added filtering of "toCreate" records when member fails to insert. We should not try inserting related members_labels/members_stripe_customers/members_stripe_customer_subscriptions records because they would definitely fail insertion without associated member record
2020-08-20 17:41:47 +12:00
Nazar Gargol
2e769e3122 Fixed handling for Stripe connected members import
no issue

- When stripe is disconnected and there are Stripe-connected records present in imported set they should not be processed and proper error should be thrown
2020-08-20 17:08:19 +12:00
Nazar Gargol
4498b4624a Fixed label serialization in members bulk importer
no issue

- After a refactor logic was missing trimming logic and handling for empty labels
2020-08-20 16:43:42 +12:00
Nazar Gargol
32fd913ac3 Fixed default subscribed value for member model in the importer
no issue

- The default value should not be a string but rather a boolean
2020-08-19 23:42:48 +12:00
Kevin Ansfield
d8e319af88 🐛 Fixed card spacing and caption styling for member emails in Outlook
closes https://github.com/TryGhost/Ghost/issues/12139

- once the email content has been rendered in the post serializer, perform some whole-content transformation of `figure` and `figcaption` to `div` using cheerio
- juiced will have already inlined the elements styles so there's no need to adjust the template's stylesheet
2020-08-19 12:34:14 +01:00
Nazar Gargol
616767d0ec Fixed members bulk import's createdBy field assignment
no issue

- Method signature has been changed with cada88975a, the call needed an update as well
2020-08-19 17:14:45 +12:00
Daniel Lockyer
78fe4f8bf9 Updated Ghost-Admin to v3.30.0 2020-08-18 16:35:02 +01:00
Jeremy Davidson
681c96b5e9
🐛 Fixed bookmark card author/publisher spacing in member emails (#12134)
closes #12078

- Root cause was that pseudo class .kg-bookmark-author:after was not getting inlined to email newsletter or its preview.
  - That is where the margin and bullet-point content are added between author and publisher.
- Dependency juice supports an option inlinePseudoElements which is false by default.
- Fix was to set inlinePseudoElements to true when serializing post email.

Co-authored-by: Jeremy Davidson <jeremy@crossingcontour.com>
2020-08-18 16:18:44 +01:00
Hannah Wolfe
9f291b2d10 Improved testmode logging of shutdown status
- testmode allows us to test Ghost's behaviour with long running requests and jobs
- moved all the testmode logging to only happen if Ghost starts successfully to make it clearer what is happening
- added a (currently very dirty) bit of code to fetch the jobService and output the queue status giving us a similar
     output for both open connections and jobs
- all of this allows for debugging exactly what Ghost is doing whilst it is processing a shutdown request
2020-08-18 15:48:42 +01:00
Kevin Ansfield
cefcdad090 🐛 Fixed emails appearing very wide in Outlook and improved email image sizes
closes https://github.com/TryGhost/Ghost/issues/11536

- bumps `@tryghost/kg-default-cards`
  - image and gallery cards now output `width/height` attributes on `img` elements with a max width of 600px
  - uses resized images where possible to keep email weight down
- adds `height: auto` style to image card images so that the `height` attribute does not cause distortion at smaller screen widths
2020-08-18 15:33:54 +01:00
Fabien 'egg' O'Carroll
cada88975a
Updated bulk importer to improve performance (#12128)
no-issue

* Added bulkAdd method to Member,Customer&Subscription model
  This allows us to keep the db access in the model layer
* Updated @tryghost/members-api to 0.27.2
  This includes fixes for rate-limiting of requests, and exposes necessary
  Stripe methods for creating customers and complimentary subscriptions,
  without affecting the database.
* Refactored importer to parallelise tasks where possible
  By parallelising our tasks we are able to improve the speed at which the
  entire import completes.
2020-08-18 13:39:45 +01:00
Daniel Lockyer
3a46aafcbb Merged 3.29.2 into master
v3.29.2

* tag '3.29.2':
  v3.29.2
  Updated Ghost-Admin to v3.29.2
  🐛 Fixed failing migration from <2.34 to 3.29
  🐛 Fixed published time and modified time for structured data (#12085)
  🐛 Fixed meta data when using tag data in collection (#12137)
  Updated Stripe JS to load async in head
  🐛 Removed [http://url/] output in member email preview text
  🐛 Fixed apostrophes not displaying correctly in Outlook for member emails
2020-08-18 12:17:35 +01:00
Daniel Lockyer
0a65471008 Updated Ghost-Admin to v3.29.2 2020-08-18 12:15:49 +01:00
Kevin Ansfield
69612e38ff 🐛 Fixed failing migration from <2.34 to 3.29
closes https://github.com/TryGhost/Ghost/issues/12129

Migrations were failing when upgrading from a version that didn't include stripe member subscriptions table. The `up` migration was checking for the existence of a unique index that was added to the schema in 3.29 but it was using the wrong variable name which meant that it would never return true for an existing index. For most migrations this worked because the index existed but when through 2.34 the table was created from scratch and did included the index at that point.

- fixed variable name and re-ordered variable assignment for better code locality that would have made the typo more visible
2020-08-18 11:34:12 +01:00
Roshan Dash
c66304f66a 🐛 Fixed published time and modified time for structured data (#12085)
closes #12059

- Published Time and Modified Time were not populating for 'page' context because it is an extension of 'post' and hence there was no context 'page'. 
- Fixed it by using the common contextObject & `getContextObject` utility. 
- Should also fix some other missing parameters.
2020-08-18 11:34:06 +01:00
Fabien 'egg' O'Carroll
993e45bd88 🐛 Fixed meta data when using tag data in collection (#12137)
closes #12130 

When defining a collection with a tag as the data source, the metadata
was not correctly applied due to the context array not including 'tag'.

This update keeps the context management all in the same context helper
file and follows the same pattern as for posts/pages as a data source.
2020-08-18 11:34:00 +01:00
Rish
1a6eec40d1 Updated Stripe JS to load async in head
no issue

- Stripe JS is added to a theme via ghost_head if a Stripe account is connected to members enabled site
- Previously, the script was not loading async which blocked the main thread, changes the script load to async to avoid rendering block
- Members script is already being loaded with `defer` so does not block the main thread
2020-08-18 11:33:54 +01:00
Kevin Ansfield
f8f0f76788 🐛 Removed [http://url/] output in member email preview text
no issue

- we output the post excerpt in a hidden div in the email template so that email clients pick it up as the "preview" text when listing emails
- when no custom excerpt is provided the preview text is grabbed from post.excerpt which is the first 500 chars of the post.plaintext value
- post.plaintext formats links as "Link [http://url/]" which is unwanted in html email previews

- add a basic replacement to the post email serializer to remove any `[http://url/]` occurrences from the post excerpt before rendering the email content
2020-08-18 11:33:48 +01:00
Kevin Ansfield
f272b3fa5c 🐛 Fixed apostrophes not displaying correctly in Outlook for member emails
refs https://github.com/TryGhost/Ghost/issues/11536

- Outlook supports `&#39;` as a special char for apostrophes but not `&#apos;` which is what cheerio/juiced render
- adds a basic string placement to the email serializer to switch to the older style of special char
2020-08-18 11:33:43 +01:00
Rishabh Garg
2bac2c67cc
Granted Admin users ability to impersonate member (#12132)
refs #12126

- Adds migration to add impersonation permission to administrators
- Adds default permission fixture to allow administrators to read member impersonation urls
- Allows administrators to create member impersonation magic links
2020-08-18 14:17:56 +05:30
Nazar Gargol
6ca51eae29 Fixed member import csv validator for labels
no issue

- Labels can have a null value when there is none defined in the imported file. Such records fails for "string" type in JSON schema
2020-08-18 15:25:04 +12:00
Kevin Ansfield
752c1022f9 🐛 Fixed failing migration from <2.34 to 3.29
closes https://github.com/TryGhost/Ghost/issues/12129

Migrations were failing when upgrading from a version that didn't include stripe member subscriptions table. The `up` migration was checking for the existence of a unique index that was added to the schema in 3.29 but it was using the wrong variable name which meant that it would never return true for an existing index. For most migrations this worked because the index existed but when through 2.34 the table was created from scratch and did included the index at that point.

- fixed variable name and re-ordered variable assignment for better code locality that would have made the typo more visible
2020-08-17 18:31:00 +01:00
Roshan Dash
c81d11b910
🐛 Fixed published time and modified time for structured data (#12085)
closes #12059

- Published Time and Modified Time were not populating for 'page' context because it is an extension of 'post' and hence there was no context 'page'. 
- Fixed it by using the common contextObject & `getContextObject` utility. 
- Should also fix some other missing parameters.
2020-08-17 15:52:31 +01:00
Fabien 'egg' O'Carroll
3430c47725
🐛 Fixed meta data when using tag data in collection (#12137)
closes #12130 

When defining a collection with a tag as the data source, the metadata
was not correctly applied due to the context array not including 'tag'.

This update keeps the context management all in the same context helper
file and follows the same pattern as for posts/pages as a data source.
2020-08-17 15:21:09 +01:00
naz
51c9a50c4f
Added validation layer to members import endpoint (#12131)
no issue

- Additional validation is needed for imported data because in case of bulk insertions (through knex) we bypass model layer validation - this could lead to invalid data in the database, which would be hard to fix.
- Chose validation method we use for other endpoints - through JSON Schema. It proved to be very performant (200ms overhead for 50k records). When comparing it with iterative method (validating each record separately) this was adding about 17s of overhead.
- Refactored returned values from "sanitizeInput" method to encapsulate more logic so that the caller doesn't have to calculate amount of invalid records and deal with error types
- Whole sanitizeInput method could now be easily extracted into separate module (somewhere close to members importer)
- Bumped members-csv package. It is meant to handle empty string values - '' and null, which should allow validating member records more consistently!
2020-08-17 23:28:57 +12:00
Rish
c6cfadbfb7 Updated Stripe JS to load async in head
no issue

- Stripe JS is added to a theme via ghost_head if a Stripe account is connected to members enabled site
- Previously, the script was not loading async which blocked the main thread, changes the script load to async to avoid rendering block
- Members script is already being loaded with `defer` so does not block the main thread
2020-08-17 10:16:42 +05:30
Daniel Lockyer
046b5540fe Removed workaround for intermittent ghost-server logging
- upon further testing, this started to not work again so I'm removing
  this horrible hack
- it could be Bunyan playing around with us, and the server is actually
  shutting down
2020-08-13 18:45:51 +01:00
Daniel Lockyer
d4f1274ba5 Returned empty promises in ghost-server
- I have no idea why this is required and I really don't like it
- this change looks superfluous but I had problems with Ghost not
  displaying shutting down messages properly and this is the only
  thing that got it working
- the async-awaits were getting stuck in `await this._cleanup()`,
  and then wouldn't enter into the `finally` block
- oddly, this was only reproducible when a job was running, it would
  shut down normally otherwise
- if I find the solution in the mean time, I'll happily get rid of this
2020-08-13 17:26:33 +01:00
Nazar Gargol
ef41c57974 Added test coverage for invalid member import emails
no issue

- This test should serve as a control for new batched importer
2020-08-13 22:14:52 +12:00
Nazar Gargol
a1e6b8f898 Updated members import sanitization
no issue

- Updated sanitization logic to be self contained and return sanitieze input along with error stats
- This should give a nice place for validations to fit in
2020-08-13 22:12:20 +12:00
Nazar Gargol
4126187df3 Fixed parameter naming for members importer 2020-08-13 21:08:34 +12:00
Nazar Gargol
d2fbe327e4 Update Stripe concurrency in members importer
no issue

- There were many failed import records due to rate-limit errors. With concurrency of 9 imports go through with 100% success
- Would need to verify these limits with live API to make the most of it
2020-08-13 21:04:57 +12:00
Nazar Gargol
78b4dff656 Moved batching logic inside the members importer module
no issue

- This way importer is more self contained and controller logic doesn't have to know about batch sizes and other unecessary variables
2020-08-13 20:31:11 +12:00
Kevin Ansfield
c434666ba2 🐛 Removed [http://url/] output in member email preview text
no issue

- we output the post excerpt in a hidden div in the email template so that email clients pick it up as the "preview" text when listing emails
- when no custom excerpt is provided the preview text is grabbed from post.excerpt which is the first 500 chars of the post.plaintext value
- post.plaintext formats links as "Link [http://url/]" which is unwanted in html email previews

- add a basic replacement to the post email serializer to remove any `[http://url/]` occurrences from the post excerpt before rendering the email content
2020-08-12 20:14:06 +01:00
Kevin Ansfield
163092f377 🐛 Fixed apostrophes not displaying correctly in Outlook for member emails
refs https://github.com/TryGhost/Ghost/issues/11536

- Outlook supports `&#39;` as a special char for apostrophes but not `&#apos;` which is what cheerio/juiced render
- adds a basic string placement to the email serializer to switch to the older style of special char
2020-08-12 20:01:42 +01:00
Kevin Ansfield
c7ff4c9e93 Moved email sending to the background job queue
no issue

- moves the meat of `pendingEmailHandler()` code into a new function `sendEmailJob()` that is passed over to the new job service
- lets the server keep processing email generation and sending when it receives a shutdown request rather than halting processing mid-send and ending up in a partial state
2020-08-12 17:02:14 +01:00
Fabien 'egg' O'Carroll
1294e3f92c
Replaced all usage of member models with members-api (#12117)
no-issue

* Added stripeSubscriptions relation to member model

This allows us to fetch the subscriptions for a member via standard
model usage, e.g. `withRelated: ['stripeSubscriptions']` rather than
offloading to loops and `decorateWithSubscriptions` functions, this is
more performant and less non-standard than the existing method.

* Updated serialize methods to match existing format

The current usage of `decorateWithSubscriptions` and the usage of
members throughout the codebase has a subscriptions array on a stripe
object on the member, this ensures that when we serialize members to
JSON that we are using the same format.

There is definitely room to change this in future, but this is an
attempt to create as few breaking changes as possible.

* Installed @tryghost/members-api@0.26.0

This includes the required API changes so that everywhere can use
members-api directly rather than models and/or helper methods
2020-08-12 14:17:44 +01:00
Nazar Gargol
c696d715c1 Extracted batched member import into separate module
no issue

- The code in controller was becoming hard to reason about.
- Having a single module shows exactly how many dependencies are there to do an import for single batch.
- Having a separate module would make it easier to extract into it's own package in Members monorepo
2020-08-12 20:18:30 +12:00
Nazar Gargol
7d3f6e32ca Addes sort order support for imported members
no issue

- Member's labels have to have sort_order assigned when added/edited. This was lacking from batched importer.
- Implementation is based on logic used in model's base - e484709e73/core/server/models/base/index.js (L81-L86)
2020-08-12 16:34:15 +12:00
Hannah Wolfe
bbbd011074 Added job-manager & wired up shutdown and testmode
- Bottom line - we need to manage shutting down gracefully when doing long-running tasks
- To achieve that, we're going to use job queues

In this commit:
- added new @tryghost/job-manager dependency
- added a minimal job service, that handles in passing things like logging and (maybe later) config
- job service is wired up to server shutdown, so that the queue finishes before the server exits
- also added a new job endpoint to testmode so that it's easy to test job behaviour without needing to do real work
2020-08-11 21:31:34 +01:00
Hannah Wolfe
624206b6d7 Improved ghost-server with async/await
- Using consistent patterns for shutdown and stop
- Make it clear what each method does
- Use async/await to make the code more readable and simple
- This lays groundwork for having more cleanup tasks in stop than just server.stop()
2020-08-11 19:42:37 +01:00
Daniel Lockyer
5b471e1bbe Extracted promise libs and history into @tryghost/promise
- deleted files under `core/server/lib/promise` and related test files
- added `@tryghost/promise` as a dependency
- fixed all local requires to point to the new package
2020-08-11 18:44:21 +01:00
Daniel Lockyer
c9a5b28669 Extracted core/server/lib/security to @tryghost/security package
- code and tests were extracted out to this package
- deletes these files
- replaces all local requires, and adds it as a dependency
2020-08-11 14:06:50 +01:00
Daniel Lockyer
8799feb801 Replaced constants file with @tryghost/constants
- extracted constants file into a new package
- replaced all local requires of the file with new package
2020-08-11 12:51:16 +01:00
Daniel Lockyer
1950d54c07 Updated Ghost-Admin to v3.29.1 2020-08-11 10:46:42 +01:00
Hannah Wolfe
e84621d6ef 🐛 Readded missing server.start event
closes #12118

- server.start was mistakenly removed in 71f02d25e9
- it is used for loading themes (and other things) and is critical
- added tests to prevent this regressing again in future
2020-08-11 10:09:24 +01:00
Nazar Gargol
8a7e00c413 Enabled batched members import method through enableDeveloperExperiments flag
no issue

- Allows early testing of batched import method
2020-08-11 18:52:37 +12:00
Nazar Gargol
bbcc0f5178 Added batched members import API method
no issue

- New Member API batched import is meant to be a substitution to current import
with improved performance while keeping same behaviore. Current
import processes 1 record at a time using internal API calls and times
out consistently when large number of members has to be imported (~10k
records without Stripe).
- New import's aim is to improve performance and process >50K
records without timing out both with and without Stripe connected
members
- Batched import can be conceptually devided into 3 stages which have
their own ways to improve performance:
  1. labels - can be at current performance as number of
labels is usually small, but could also be improved through batching
  2. member records + member<->labels relations - these could
be performed as batched inserts into the database
  3. Stripe connections - most challanging bottleneck to solve because
API request are slow by it's nature and have to deal with rate limits of
Stripe's API itself
- It's a heavy WIP, with lots of known pitfalls which are marked with
TODOs. Will be solved iteratively through time untill the method can be
declared stable
- The new batched import method will be hidden behind 'enableDeveloperExperiments' flag to
allow early testing
2020-08-11 18:31:31 +12:00
Daniel Lockyer
f83ee621a2 Updated Ghost-Admin to v3.29.0 2020-08-10 17:00:10 +01:00
Hannah Wolfe
b0512f2c25 Added a server testmode to help test behaviour
- A simple way to test behaviours without having to do complex interactions to e.g. generate errors or slow requests
- Makes it easier to test the new shutdown behaviour, among other things
2020-08-10 16:38:49 +01:00
Hannah Wolfe
a9c6e081cf Added an additional log to notify shutdown start
- there can now be quite a big delay between SIGINT/TERM being received and shutdown finishing
- add an extra log message to acknowledge the SIGINT/TERM to facilitate debugging and just be clear
2020-08-10 14:53:05 +01:00
Daniel Lockyer
226dc32ec5 Fixed default shutdown timeout
- should be 60s (60000ms), not 600s (600000ms)
2020-08-10 13:00:02 +01:00
Hannah Wolfe
19e3b70c7a Added stoppable for graceful shutdown of requests
- stopppable is a dependency that handles closing connections properly, which server.close does not
    - active connections are allowed to complete what they are doing
    - idle connections are closed
    - no new connections are allowed
- we call stoppable in stop() instead of server.close so that idle connections don't hold the server open
- calling await stop() from shutdown then ensures that we have a consistent experience of stop
- all together this allows ghost to shutdown gracefully when there are long-running requests
- @TODO: handle graceful shutdown of long-running processes
- @TODO: consider do we need to send 503s whilst the server is shutting down?
2020-08-10 11:46:36 +01:00
Hannah Wolfe
e72cc193c6 Refactored GhostServer to use a class
- use more up-to-date and less confusing syntax
- makes it easier to use async/await as well
2020-08-10 08:53:09 +01:00
Hannah Wolfe
2289b7c0f7 Moved sig event and stop message handling
- none of this should be inside the start message handling
- move to inside the start function, where it's clearer and makes more sense
2020-08-10 08:53:09 +01:00
Hannah Wolfe
31981d086b Updated stop messages to have a consistent interface
- changed method to logStopMessages, as we use start and stop, not start and shutdown
- changed logStopMesasges to output the "proper" messages and use this method consistently - the closing connections message isn't really useful
- changed uptime message to always be output cos I can't see a case where there isn't interesting/useful
2020-08-10 08:53:09 +01:00
Hannah Wolfe
f8cdaf0c24 Removed unused connection handling + restart method
- Connection handling is legacy code added in 1438278ce4
- Although we were tracking all connections in memory, we weren't actually closing any because stop isn't called
- This (and restart) were both added as part of the now long-deprecated system for using Ghost directly as an npm module
- If we want to close connections cleanly, we should use a tool to do this
2020-08-09 17:31:01 +01:00
Hannah Wolfe
71f02d25e9 Refactored server announce functions to be clearer
- the announce functions exist for the purpose of communicating with Ghost CLI
- the functions were called announceServerStart and announceServerStopped, which implies they tell Ghost CLI when the server starts and stops
- however, the true intention / purpose of these functions is to:
    - either tell Ghost CLI when Ghost has successfully booted (e.g. is ready to serve requests)
    - or tell Ghost CLI when the server failed to boot, and report the error so that Ghost CLI can communicate it to the user
- therefore, I've refactored the old functions into 1 function to make it clearer they do the same job, but with 2 different states
- also added some tests :D
2020-08-09 17:25:15 +01:00
Hannah Wolfe
c577007afe Update dependency @tryghost/bootstrap-socket to v0.2.0
- includes API changes
2020-08-09 17:22:27 +01:00
Hannah Wolfe
b765a30ee9 Moved bootstap-socket code to @tryghost/bootstrap-socket
- this code has zero deps and is much easier to reason about standalone
2020-08-07 18:43:46 +01:00
Hannah Wolfe
b09504e7cc Moved bootstrap socket logic out of server
- this logic is a dependency rather than part of the server code
- moved it out ready to be moved elsewhere, doesn't need to be here
2020-08-07 16:27:45 +01:00
Hannah Wolfe
028d3fc88b Added missing debug info for IPC on start
refs 022a433e56

- noticed I missed adding debug info in one place (not that it's used, just for consistency)
- also made the SIGINT/TERM code slightly more readable
2020-08-06 20:39:49 +01:00
Kevin Ansfield
2efcf94645
Improved performance of sending newsletter emails (#12091)
no-issue

- switch from `membersService.api.members.list` to using bookshelf `Member.findPage()` with the `{paid: true}` filter to avoid per-member queries (N+1) to decorate members with subscriptions and a heavy post-fetch filter via `contentGating`
- add concurrency to the Mailgun API requests in `bulk-email` service to reduce overall time submitting API requests
- add debug statements with timing output for easier measurements
2020-08-06 15:19:39 +02:00
Kevin Ansfield
490f9787fa
Added migration to create indexes and constraints for member tables (#12108)
no issue

For large numbers of members we're unable to perform queries for free/paid members in a reasonable time without indexes. Bulk delete is also very slow when looping through bookshelf models and having bookshelf manage deletion of child records, adding `ON DELETE CASCADE` to the foreign key indexes moves child deletion to the DB level and allows for performant bulk delete queries.

- migrations only run on mysql because sqlite does not support altering tables
- adds foreign key indexes and constraints with 'ON DELETE CASCADE' for members_labels, members_stripe_customers, and members_stripe_customers_subscriptions
2020-08-06 14:57:05 +02:00
Fabien 'egg' O'Carroll
d3384975da
Cleaned up members_stripe_* tables on MySQL (#12103)
refs #12100

For performance reasons we want to add foreign key and unique constraints
to the members_stripe_* tables so we can utilised cascading deletes and 
joins across the tables when querying.

In order to do this we must first ensure that:
- There are no duplicate entries in the `subscription_id` or `customer_id` columns
- There are no orphaned rows in the subscription or customers tables

If the first is not true, the unique constraint will fail, and if the second is not true,
the foreign key constraint will fail.

As we are only adding the indexes to existing MySQL databases at this point, the
cleanup migrations will also only be done for existing MySQL databases too.

The migrations for removing orphaned rows splits the deletion into a `SELECT`
followed by a `WHERE IN` to avoid the database "optimising" the query into a
`JOIN` which ends up taking much longer due to the lack of indexes.
2020-08-06 11:13:41 +02:00
Fabien 'egg' O'Carroll
d15446593a
Added support for ON DELETE CASCADE to the schema (#12105)
no-issue

We are in the process of creating migrations to add foreign key constraints
and cascading deletes to the members_stripe_* tables to make listing members
and deleting members faster. As well as the migrations we need to update the
database schema so that new installations have the correct indexes and constraints.

Co-authored-by: Kevin Ansfield <kevin@lookingsideways.co.uk>
2020-08-05 13:20:30 +02:00
Daniel Lockyer
226fa76c51 Updated Ghost-Admin to v3.28.0 2020-08-04 11:23:40 +01:00
Nazar Gargol
60ae9e82f9 Fixed integration_id assignment for webhook when creating through API key auth
refs 173e3292fa

- The bug was initially introduced in referenced commit. When request is done with `api_key` context, there should always be an `integration` object associated with it - 71c17539d8/core/server/services/permissions/parse-context.js (L36) . An `id` from `context.integration` not `context.api_key` has to be assigned to newly created webhook!
- The webhooks API is about to be declared stable in upcoming release, so no migration will be done
2020-08-04 16:43:24 +12:00
Gerardo Lopez Dueñas
275bf41ba3
Fixed bookmark card image size in fastmail (#11970)
closes #11907

The image in the bookmark card was being shown out of the bounds of
the card because of a general style `height: auto !important`.

I added a new `max-height` property to the image to avoid exceeding
parent height.
2020-08-03 18:38:45 +01:00
Rish
f770e9f2ad 🐛 Fixed incorrect member gating on custom static page routes
no issue

- Member gating on post/page content is done using member context on request's frame
- Custom static page route was incorrectly setting the member context as `members` instead of `member`, which caused logged-in members to still not able to see content while logged in as a member
- Fix corrects the member context for these routes to use `member` as expected
2020-08-03 17:28:55 +05:30
Nazar Gargol
2a4ad0e10f Updated stability index comments around Admin API v2 endpoints
refs 067d2eb614

- v2 API has been in maintenance mode since v3 has been released, updated comments accordingly
2020-08-03 23:46:23 +12:00
Nazar Gargol
067d2eb614 Moved canary/v3 webhooks API to "stable" group
no issue

- Webhooks API has been stabilized with latest changes and there are no breaking changes planned for v3. The change has strictly "informative" purpose
- Changed variable naming from "whitelisted" to "allowlisted" to follow updated naming convention (refs. https://mysqlhighavailability.com/mysql-terminology-updates/)
2020-08-03 23:40:15 +12:00
Nazar Gargol
b76a6a1eee Reduced error level to default for not found post error
refs #12064

- `critical` is meant to be something unpredictable like internal error, something worthy attention, as described in Ignition -3439456d94/README.md (list-of-errors)
- This error level was introduced with - this PR https://github.com/TryGhost/Ghost/pull/9426, but there is no context provided why this specific value was used. Assuming it's an outdated value as 'not found' is nowhere to be treated in any special way
2020-08-03 23:16:53 +12:00
Nazar Gargol
1b449f4f53 🐛 Fixed 500 error in webhooks API when modifying non-existing webhooks
closes #12064

- Handled permission check bug by returning 404, same way it is returned in other permissions related places when handling non-existing resource. Example - 60907a7ae4/core/server/models/relations/authors.js (L355-L358)
2020-08-03 23:08:47 +12:00
Kevin Ansfield
29d94e7814 Fixed mailgun config not allowing custom hosts with ports
no issue

- `mailgun()` expects the `host` option not to include a port but `url.host` will include the port, we instead want to use `url.hostname` which skips the port
2020-07-30 17:28:51 +01:00
Kevin Ansfield
577a934f53
Removed DISTINCT from member labels association query (#12088)
no issue

- bookshelf adds `DISTINCT` to any relation query that does not have an explicit `columns` statement
- when measuring the impact of `DISTINCT` on the eager-loading association query when listing members using `{withRelated: 'labels'}`, it can be 2x slower with no index on the sort_order column or 4x slower with an index on sort_order
2020-07-29 12:50:22 +01:00
Nazar Gargol
ef6586bfdc Fixed failing members import with label associations
no issue

- When processing entries with new labels in parallel Bookshelf relations is trying to create them which caused unique key constraints to fail. To avoid the failure, all labels should be pre-created before proceeding with creating members
2020-07-28 22:37:48 +12:00
Fabien O'Carroll
5c9188b1e5 Updated Ghost-Admin to v3.27.0 2020-07-27 18:38:26 +02:00
Hannah Wolfe
436db4ec3b Updated host limit message to be clearer
- limits are based on total members not number of members that will be emailed
2020-07-26 20:49:30 +01:00
Hannah Wolfe
92446d85ea Changed member limit to be DRY & use raw query
- Member limit code was duplicated in 2 places unnecessarily
- Also used member api code that fetched members and subscriptions fully hyrated when we only need a count
- Using a raw query significantly improves performance here
2020-07-26 20:49:30 +01:00
Kevin Ansfield
14c6968388 Fixed auto-generated label not being attached to members during import
no issue

- label was being created and returned in the response OK but it was missing from the set of labels that get added to members during import
2020-07-26 18:30:58 +01:00
Fabien O'Carroll
c46475f5be Supported optionally cancelling subscriptions on delete
no-issue

This updates the Admin API Member resource to *not* cancel subscriptions
by default, and adds a `cancel` option. This can be used over HTTP by
including a `cancel=true` query parameter.
2020-07-24 16:01:06 +02:00
Fabien O'Carroll
92151a7b5d Added default options to transactional model methods
no-issue

This protects against calling model methods without passing an options
object
2020-07-24 15:31:48 +02:00
Hannah Wolfe
3491e60c9d Added config to send bulk email in testmode
- mailgun has a testmode flag we can use to get email to be accepted but not delivered
- this is useful for developers testing general bulk email code - not for users - so it is only available via config
2020-07-24 11:55:34 +01:00
Fabien 'egg' O'Carroll
5144a0e09c
Updated Member model to cascade on destroy (#12077)
no-issue

Up until now we have left orphaned rows in members_stripe_* tables when
a member is deleted, this updates the destroy method so that we cascade
and remove any MemberStripeCustomer and StripeCustomerSubscription
models related to the Member.

This also adds regression tests for the new functionality as well as to
confirm the existing functionality of cascading to the members_labels
join table

This adds the relations of Subscription->Customer & Customer->Member
2020-07-23 18:21:10 +02:00
Rish
2ac69e637e Added publication icon to members site data
refs https://github.com/TryGhost/members.js/issues/72

- Portal is using using publication logo from settings for signup/signin pages
- Instead, we are switching to using publication icon from settings, which also needs to be passed in site data API
2020-07-23 17:04:44 +05:30
Kevin Ansfield
8ceabbcfba Added automatic labelling of all members added during an import
no issue

Having all members created during an import labelled with a specific "import label" is useful for later operations such as bulk delete/edit or simply recording how and when a member was created.

- automatically create a label with the date/time the members CSV import occurred and assign it to all imported members
- return the import label data in the API response so that clients can react accordingly such as automatically filtering the members list by the label once an import finishes
2020-07-22 17:52:43 +01:00
Daniel Lockyer
5e45cd9155 Updated Ghost-Admin to v3.26.1 2020-07-22 11:56:55 +01:00
Fabien 'egg' O'Carroll
69080824de
🐛 Ensured webhooks are created once (#12075)
refs #12074

Since we've split members settings into multiple keys the
reconfiguration of the members-api has been happening in quick
succession as the stripe_connect_* settings are all set at once.

This debounces the call to reconfigure the members-api so that we only
need to instantiate it once.
2020-07-22 12:27:05 +02:00
Daniel Lockyer
38f3e3ff03 Updated Ghost-Admin to v3.26.0 2020-07-21 13:45:56 +01:00
Hannah Wolfe
995a4eb46e 🐛 Fixed edit link for staff members
closes #12069

- the url in admin changed from team to staff, and wasn't updated in the frontend
2020-07-21 12:28:50 +01:00
Rish
3d164d222b Updated error name check in GhostMailer
no issue
refs e8511d0568

- Adds extra check for empty `err` object while checking error name for status code in GhostMailer
2020-07-21 01:08:19 +05:30
Rish
e8511d0568 Updated error code for incorrect recipients in GhostMailer
no issue

- By default, GhostMailer throws EmailError with statusCode as `500` for any failure in sending mail
- In case of failure due to `RecipientError`, status code as now correctly sent as `400` as its a bad request and not an error we can't handle.
2020-07-21 01:05:21 +05:30
Fabien 'egg' O'Carroll
6232981be7
🐛 Fixed importing Stripe Plans with amount 0 (#12062)
closes #12049

Stripe plans used to default to 0, and our new validation of plan
amounts were causing issues when importing from an older version of
Ghost, this updates the validation to be skipped when importing.

- Added regression test for importing plans
2020-07-20 14:59:23 +02:00
Daniel Lockyer
d261f88456 Fixed typos in webhook error message
no issue
2020-07-20 09:05:56 +01:00
Rish
90b39fbb9a Updated status and error message for newsletter email failures
refs https://github.com/TryGhost/Ghost/issues/11971

- Added statusCode from bulk email provider to API response
- Updated error messages for different bulk email(mailgun) failure states
- Added `context` to preview mail API error message with mail provider's error message
2020-07-17 13:54:09 +05:30
Nazar Gargol
a520cdad0b Added JSON Schema validations to Webhooks Admin API v3
closes #12033

- Added webhooks schemas and definitions.
- Added validation checking if integration_id is present when using session auth. This is needed to prevent orphan webhooks.
- Integrated webhook schemas into frame's validation layer.
- Added isLowerCase ajv keyword support. This is needed to be able to do isLowerCase validation using JSON Schema for webhooks.
2020-07-17 17:37:14 +12:00
Fabien 'egg' O'Carroll
8f660c3259
Improved settings validation (#12048)
closes #12001

* Moved settings validation to the model

This moves the settings validation out of the validation file and into
the model, as it is _only_ used there.

It also sets us up in the future for custom validators on individual
settings.

* Improved validation of stripe_plans setting

- Checks `interval` is a valid string
- Checks `name` & `currency` are strings

* Moved stripe key validation into model

The stripe key settings are all nullable and the regex validation fails
when the input is `null`. Rather than reworking the entirety of how we
validate with default-settings validation objects, this moves the
validation into methods on the Settings model.

* Added tests for new setting validations

Adds tests for both valid and invalid settings, as well as helpers
making future tests easier and less repetitive
2020-07-15 17:11:27 +02:00
Fabien O'Carroll
8cedbdf07c Fixed version for amp_gtag_id setting migration
refs #11980

This was added to the 3.25 directory _after_ 3.25.0 was released, thanks
to @daniellockyer for the catch!
2020-07-15 15:33:37 +02:00
Fabien O'Carroll
23bf432514 Updated AMP output to include Google Analytics
refs #11980

This adds a new handlebars helper {{amp_analytics}} which outputs the
GA configuration script if the amp_gtag_id setting is present.

We also update {{amp_components}} to output the GA script in the head if
the setting is present
2020-07-15 13:43:25 +02:00
Fabien O'Carroll
48eca5f618 Added amp_gtag_id setting migration & default
refs #11980

This lays the ground for the rest of the work surrounding Google
Analytics in AMP.
2020-07-15 13:43:25 +02:00
Nazar Gargol
ced3e28d60 🐛 Fixed cache invalidation for published posts with no content
closes #12015
refs 95880dddeb

- The bug was caused by falsy plaintext field assignment to empty string `''` when the html content was `null`. Because of the `setEmptyValuesToNull` function (referenced commit), there is no sense to assign empty string value to plaintext property, because it would still end up being `null`
- The `''` -> `null` conversion was confusing the model layer to think that some fields were changed, where in reality none did. This in turn lead to a bug with falsy cache invalidation
2020-07-15 17:41:24 +12:00
Nazar Gargol
a75e1e27db 🐛 Fixed cache invalidation when editing post_meta fields
closes #12016

- The change detection didn't work when editing post_meta fileds because we only check current model's `_changed` fields when performing `wasChanged()` check
- A solution was adding change tracking of post_meta relation to currently edited post model and overloading `wasChanged` method to check these fields as well
2020-07-14 23:48:36 +12:00
Daniel Lockyer
8c45eca2c9 Updated Ghost-Admin to v3.25.0 2020-07-13 14:52:36 +01:00
Rishabh Garg
ead073ad68
Updated sender email to existing "from-address" (#12044)
refs #12043

- On updating From-address from Members settings in Labs, we send a confirmation email to the updated address with magic link for verification
- Previously, no explicit sender email was being set for this so fallback config address was used
- This updates the sender address to use the current from address for "from-address" update emails
2020-07-13 17:57:31 +05:30
Fabien O'Carroll
06c7152f4b 🐛 Fixed Stripe webhooks for subdirectory setups
no-issue

- Added breaking test for webhook url including subdirectory

- Previously the webhook handler URL was generated incorrectly when
  running Ghost on a subdirectory, appending the path to the root of the
  host, this fix ensures that the subdirectory is included before the
  path.
2020-07-10 17:43:46 +02:00
Fabien O'Carroll
832ecad43b Improved handling of Stripe webhooks on boot
no-issue

This version of members-api includes changes to how webhooks are
managed, previously they would be deleted and recreated on every boot of
Ghost. Now they are created and the secret is persisted, on boot the
webhook is updated to the most current url and events. If the api
version is wrong or the update fails, the webhook is deleted and
recreated and the settings updated.

 - Installed @tryghost/members-api@0.24.0
 - Updated config to work with 0.24.0
2020-07-10 17:43:46 +02:00
Fabien 'egg' O'Carroll
20a95de8f7
Added new stripe webhook settings (#12028)
no-issue

They will be used to store webhook information so that we can persist it between
boots and simplify the creation process of webhooks in members
2020-07-10 16:26:06 +02:00
Daniel Lockyer
63ccbb817d Updated Ghost-Admin to v3.24.0 2020-07-10 14:26:50 +01:00
Rish
5fbc40430b Added migration to update group for new portal settings
closes https://github.com/TryGhost/Ghost/issues/12026

- 3 new portal settings were added in `3.23` in default settings - `portal_button_style`, `portal_button_icon`, `portal_button_signup_text`
- New settings default to group `core` on migrating from pre 3.22 versions due to missing columns in DB
- Migration here updates the new settings to correct group
2020-07-10 18:32:54 +05:30
Fabien O'Carroll
1d8154f892 Updated ghost_head & ghost_foot to work with tags
no-issue

This adds support for tags codeinjection to work with the ghost_head and
ghost_foot helpers
2020-07-10 14:20:24 +02:00
Fabien 'egg' O'Carroll
2dd302a23e
Updated frontend meta helpers to support tag metadata (#12037)
no-issue

- `canonicalUrl`
  - Updated to use `canonical_url` & fall back to previous functionality
- `ogTitle`
  - Updated to use `og_title` and fall back to previous functionality
- `ogImage`
  - Updated to use `og_image` and fall back to previous functionality
- `ogDescription`
  - Updated to use `og_description` and fall back to previous functionality
- `twitterTitle`
  - Updated to use `twitter_title` and fall back to previous functionality
- `twitterImage`
  - Upated to use `twitter_image` and fall back to previous functionality
- `twitterDescription`
  - Updated to use `twitter_description` and fall back to previous functionality
2020-07-10 13:52:48 +02:00
Nazar Gargol
d6267340a1 Updated Content API resource ordering to be same as slugs in filter
closes #11994

- Adds support for ordering based on slug filter  that contains a slug-is-in filter. It is applied only to Content API's resources - post, page, tag, author. The order is applied in the same order in which slugs appear in the filter.
- For, example providing following  query parameter filter for any of the above resources: `?filter=slug:[kitchen-sink,bacon,chorizo]`, would filter them by these slugs and order in the same way defined in the filter
- Can be used in handlebars templates in following way: `{{#get "tags" filter="slug:[slugs,of,the,tags,in,order]"}}`
- The property conteining this new order is assigned to `autoOrder` instead of `rawOrder` intentionally. This explicit asstignment would allow distinguishing where the 'orderRaw' comes from the model or the API layer. Apart from  adding necessary context this separation makes it easier to refactor separately model layer and API specific ordering in the future
- This commit also fixes default filtering for `author` resource in Content API. The serializer was never used before as it was missing from `serializers/index.js` module.
2020-07-10 18:33:00 +12:00
Peter Zimon
0ea767ff5c Updated member already exists error message
no refs.
- changed member already exists error message from "Duplicate email address"
2020-07-09 12:13:26 +02:00
Daniel Lockyer
251c552c8a Updated Ghost-Admin to v3.23.1 2020-07-09 10:05:08 +01:00
Rish
858bbfe4a7 Fixed public membersjs script load behind dev flag
no issue

- Currently both old and new membersjs script are getting injected behind dev flag
- New script handles the current script behavior as well making its load redundant when on dev flag
- Removes old script load in favor of new
2020-07-09 11:07:01 +05:30
Matt Hanley
64da78bd6d
Updated twitter social profile default value (#12024)
no issue

- Updates default twitter profile value from `tryghost` to the correct `ghost`
2020-07-08 22:58:50 +05:30
Daniel Lockyer
97537b3010 Updated Ghost-Admin to v3.23.0 2020-07-08 15:23:23 +01:00
Fabien O'Carroll
a3f693b472 Updated tags output serialiasation for v2 API
no-issue

This ensures that the v2 API only outputs the tag properties we specify,
and doesn't include any new fields, like the new metadata columns.
2020-07-08 15:14:40 +02:00
Fabien O'Carroll
7e5292eccc Updated tags input schema for canary API
no-issue
2020-07-08 15:14:40 +02:00
Fabien 'egg' O'Carroll
e14188807d Fixed express-jwt config
no-issue

express-jwt expects an array of valid algorithms, not a single algorithm.
2020-07-08 15:09:44 +02:00
Fabien O'Carroll
a607610c3a Added new meta columns to tags table
no-issue

This prepares the database for future work, allowing tags to store metadata
2020-07-08 11:49:22 +02:00
Rish
c5d360e647 Fixed members plan currency symbol
no issue

The refactor to new settings lost the uppercasing of the currency string, this puts it back
2020-07-08 10:47:28 +05:30
Nazar Gargol
173e3292fa Fixed integration_id handling in Webhooks Admin API
no issue

- Changes introduced to both API v3 and v2
- Makes sure to use the same integration_id as authenticated integration for the webhook's data.
-  Makde it is impossible to create orphaned webhooks using token authentication
- Allowed only parent integration to edit it's children webhooks. Throwing permission error otherwise
2020-07-08 16:54:31 +12:00
Fabien O'Carroll
694d92d270 Added guards for stripe_plans setting being null
no-issue

There is concern that the settings cache can return `null` for values
which it cannot parse correctly, this just ensures that we always have
an array where we expect one
2020-07-07 13:10:17 +02:00
Rish
8675a7d317 Updated portal button setting to false
no issue

- Existing and new sites should have portal button hidden by default and need explicit switch on from Admin.
- Default value for new sites has been updated, but existing sites still have old default and portal button set as true
- Migration here resets portal button setting to false for all existing sites, and they'll need to be explicitly turn it on from admin again
2020-07-07 14:43:00 +05:30
Rish
8d989bd3c3 Updated members site data to include new portal settings
no issue

- Adds new portal button settings to members site data for portal script
- Updates settings input/output serializers to handle portal icon image url
2020-07-07 14:06:04 +05:30
Rish
c0e3c8bcfe Added new settings for portal button customization
no issue

- Adds new portal settings - `portal_button_style`, `portal_button_icon` and `portal_button_signup_text`
- New settings allows customization of portal button
- Updates tests to include new settings
2020-07-07 14:06:04 +05:30
Hannah Wolfe
8c8af3b4ac Fixed additional bracket in fixture code example
closes #11908

- The post publishing-options shows a snippet of JSON-LD in a code block, that had an extra comma and curly bracket
2020-07-06 19:49:10 +01:00
John O'Nolan
70f16f3d97
🎨 New Google AMP template (#11986)
- Refreshed the design of the default Ghost AMP template
- Updated tests
2020-07-06 17:09:24 +01:00
Daniel Lockyer
8956a9df35 Merged 3.22.2 into master
* tag '3.22.2':
  v3.22.2
  Updated Ghost-Admin to v3.22.2
  Emitted all settings events on reinit of cache (#12012)
  🐛 Updated access to be true by default in v3 API
  Hardened members subscription migration against missing data (#12009)
2020-07-06 16:19:55 +01:00
Daniel Lockyer
f007a76ed5 Updated Ghost-Admin to v3.22.2 2020-07-06 16:17:13 +01:00
Fabien 'egg' O'Carroll
8b6ec4d922
Emitted all settings events on reinit of cache (#12012)
closes #12003

There are a few parts of Ghost that rely on the settings events being
emitted anytime a setting is changed, so that the data is kept in sync.

When a setting is renamed in a migration essentially what happens is
that the settings value is changed from a default value to its actual
value, but this does no emit an event.

Anything that is initialised before migrations have run that relies on
the events to keep it up to date will have stale data - e.g. the themes
i18n service.

This change ensures that when we `reinit` after migrations have been
run, we emit events for every setting to tell the rest of Ghost that it
has changed.
2020-07-06 17:09:43 +02:00
Hannah Wolfe
289c1b3e8a 🐛 Updated access to be true by default in v3 API
closes #11990

- access should be a members feature, but it was already accidentally exposed to the theme layer
- it has now been added to the API even if members is disabled
- access defaults to true, unless members is enabled
- when members is enabled, access is set to the currently logged in members' access
2020-07-06 16:04:29 +01:00
Fabien 'egg' O'Carroll
4a5079085a
Hardened members subscription migration against missing data (#12009)
closes #11993

We had some issues with some databases being in an unexpected state this
check for each property before using it add uses defaults when it is
missing.
2020-07-06 12:18:13 +02:00
Rish
27379b76e4 Revert "Updated default visibility for membersjs button"
This reverts commit bf80ba924f.

- New sites should have the portal enabled and shown by default when they enable members (behind dev flag currently)
2020-07-06 14:12:01 +05:30
Daniel Lockyer
f65c425786 🐛 Fixed searching for free/paid members
no issue

- when searching for paid/free members, the `members_stripe_customers`
  table would be joined into the query on `members`
- this table also has a `name` and `email` field, so both MySQL and
  SQLite would complain about ambiguous fields in the query
- the result of this would be a 500 error thrown inside Ghost, and no
  useful response to the user
- this commit explicitly chooses the `members` table to check against,
  and also adds a test for this
2020-07-06 09:09:08 +01:00
Rish
bf80ba924f Updated default visibility for membersjs button
no issue

- The membersjs default trigger button was set to be visible by default, even behind dev flag
- This changes the default visibility to false so it needs to be switched on from Admin to be visible on screen
2020-07-06 10:46:07 +05:30
Fabien O'Carroll
6ffb820137 Removed bulk_email_settings from settings
refs #10318

 - Removes old default setting
 - Adds migration for removing setting from database
 - Fixes regression tests
2020-07-03 16:49:49 +02:00
Fabien 'egg' O'Carroll
37d577fa90
Remove references to bulk email settings (#12000)
refs #10318

* Ignored bulk_email_settings when importing
* Removed custom handling for bulk_email_settings
* Removed bulk_email_settings from default-settings
* Ignored bulk_email_settings when editing settings
2020-07-03 16:30:54 +02:00
Fabien O'Carroll
957cf9fd1d Added mailgunIsConfigured prop to config API
refs #10318

This will replace the isConfig property attached to the bulk_email_settings
2020-07-03 11:48:47 +02:00
Fabien O'Carroll
5e6a4f6f7d Updated bulk-email service to use mailgun settings
refs #10318
2020-07-03 11:48:47 +02:00
Fabien 'egg' O'Carroll
9df217b04e
Added mailgun_{domain,api_key,base_url} settings (#11992)
refs #10318

 - Adds new default settings
 - Adds migration for populating settings with bulk_email_settings data
 - Fixes regression tests
2020-07-03 11:00:20 +02:00
Fabien O'Carroll
f3b4a538af Updated the validateSettings method to check stripe_plans
refs #11765

This ensures that plans will always be set to 1 of the currency
2020-07-03 10:37:13 +02:00
Fabien O'Carroll
38fce4efc5 Cleaned up data validation module
no-issue

Removes overly verbose repetition of identifiers making this easier to read
2020-07-03 10:37:13 +02:00
Fabien O'Carroll
774187ced5 Cleaned up onValidate method on settings model
no-issue

Explicit arguments and async/await make this nicer on the eyes
2020-07-03 10:37:13 +02:00
Kevin Ansfield
e9ae0d9baa 🐛 Fixed srcset and sizes attributes being rendered when image resizing unavailable
closes https://github.com/TryGhost/Ghost/issues/11944

- updates `@tryghost/image-transform` to version that exposes `canTransformFiles()` which checks for `sharp` availibility
- updates `@tryghost/kg-default-cards` to version that accepts a `canTransformImage()` method as an option
- updates our `mobiledoc` lib to pass a `canTransformImage()` function that returns false if sharp is unavailable, the image extension is not supported, or the storage engine in use does not support image transforms
- updates `populateImageSizes` to fetch image sizes when transforms are unavailable as the render/not-render is now handled in the renderer and we don't need to worry about adding size information to the mobiledoc source
2020-07-02 18:03:22 +01:00
Nazar Gargol
6902f86b40 Added field mapping support for members import CSVs
no issue

- This is a precursor work allowing client to send csv to JSON
field mapping hash as a `mapping` field parameter in the incoming
request.
2020-07-02 15:18:39 +12:00
Daniel Lockyer
27e49a4edc Updated Ghost-Admin to v3.22.1 2020-07-01 18:38:32 +01:00
Kevin Ansfield
9d7665bb43 🐛 Fixed global code injection not being output
refs https://github.com/TryGhost/Ghost/issues/10318

- settings key rename was missed in the `ghost_head` and `ghost_foot` helpers
2020-07-01 17:44:59 +01:00
Kevin Ansfield
fd4a011995
🐛 Fixed settings cache being out of sync after migrations (#11987)
refs https://github.com/TryGhost/Ghost/issues/10318

- re-initialize settings cache after migrations by shutting down to clean up event listeners then and calling `init` again
- important to ensure `db.ready` event is not emitted until settings have finished re-initializing to avoid problems with background processes using the db connection which is disconnected/re-connected or being kicked off with out-of-date settings
2020-07-01 17:16:57 +01:00
Fabien 'egg' O'Carroll
cbdf33bfb7
Fixed @price.currency_symbol helper
closes #11984

The refactor to new settings lost the uppercasing of the currency string, this puts it back
2020-07-01 13:26:30 +02:00
Daniel Lockyer
e457f2018c Updated Ghost-Admin to v3.22.0 2020-07-01 10:49:49 +01:00
Nazar Gargol
d03f674a2f Fixed failing members import tests
refs 05f6faf846

- The logic in "Stripe" error detection depended on error's message string matching, which is not a good practice in general. Had fixed it do to exact match on "context" of the error we throw internally and left more extensive comment about why things are implemented the way they are.
2020-07-01 19:03:12 +12:00
Peter Zimon
05f6faf846 Updated error messages for member import
no refs.
- refined errors for member import post-validation
2020-07-01 07:23:20 +02:00
Fabien O'Carroll
44c03eb0f4 Fixed migration exiting for single missing setting
refs #10318

`return` would break the whole loop, `continue` will continue to the
next iteration
2020-06-30 18:38:55 +02:00
Fabien O'Carroll
0074324b10 Fixed migration when brand setting is null
refs #10318

This would throw if the setting was null, we catch and use the default
instead.
2020-06-30 18:30:05 +02:00
Kevin Ansfield
9810c41dca 🐛 Fixed facebook/twitter images having relative URLS in API output
no issue

Output serializer's url util was expecting `og_image` and `twitter_image` to be top-level attributes in the `attrs` object but they are actually nested under `posts_meta`.

- updated the code to use lodash's `get/set()` so that we can work with paths for easier handling of nested objects
- fixed unit tests where the mocked data under test did not match real-world data
2020-06-30 16:40:34 +01:00
Jonas Fabisiak
58ac033b97
Updated head to only output custom favicons (#11949)
closes #11932

- as per the issue, there is no need to output a <link> tag if the favicon is the default /favicon.ico, as all browsers automatically check for this 
- instead the favicon <link> is only output if a custom favicon has been set, telling the browser to look somewhere different to its default location
- some of the tests expected 3 links in the html head, but through the favicon change there are often only 2.

Co-authored-by: RenCloud <rencloud@pop-os.localdomain>
2020-06-30 15:07:08 +01:00
Hannah Wolfe
fa91c6c954
🐛 Moved access to an API property (#11967)
closes #11574

- the current implementation of the access property has it frontend only, and wired up only in one place
- this leaves it only available in a handful of places, e.g. can't use it in a post loop or get helper
- the current implementation also fails logically if the html content of the post is blank

This fix moves the behaviour to the API

- this ensures the field is always available no matter what context you are in
- it also updates the logic to use the same membersHasAccess logic as is used to gate the post, so it's always correct

TODO: should reconsider the location of this code
2020-06-30 14:46:46 +01:00
Rish
212e39f772 Updated settings importer to handle new mapping
refs https://github.com/TryGhost/Ghost/issues/10318

- Skipped `members_subscription_settings` and `stripe_connect_integration` from settings import
- Fixed import data field filtering for core and theme groups
2020-06-30 17:38:36 +05:30
Rish
d5f68dbbc5 Updated serializers/model layer validation using settings type
refs https://github.com/TryGhost/Ghost/issues/10318

- Updates `boolean` serialization in v2/canary serializers to apply only for `boolean` type settings
- Updates `boolean` transformation in model layer `format`/`parse` to check on `boolean` type setting
- Removes error thrown on Read-only setting for settings edit endpoint
- Updates v2/canary input serializers to remove any Read-only settings (using RO flag) to avoid edits
- Added type/group mappings in the importer when pre-migration settings table import data is present
- Updates tests
2020-06-30 17:38:36 +05:30
Rish
8a50a3e9c9 Updated default settings and schema with updated types
refs https://github.com/TryGhost/Ghost/issues/10318

- Updates default settings to contain correct type and validation for each setting
- Updates `populateDefaults` to correctly insert type value for new settings
- Updates settings schema to allow only select types - `array`, `number`, `boolean`, `string`
- `object` is a temporary type allowed till we get rid of all JSON object settings
2020-06-30 17:38:36 +05:30
Fabien O'Carroll
01633470f8 Switched on Stripe Connect by default
no-issue

This flags switches off the old API Key UI and replaces it with the new
Stripe Connect flow!
2020-06-30 14:07:33 +02:00
Fabien O'Carroll
a5751341b6 Added RO flag to members_from_address
refs #10318

This was incorrectly missing before
2020-06-30 13:54:01 +02:00
Fabien O'Carroll
f2f40a5f4a Updated migrations to always set group,types,flags
refs #10318

Because settings are not populated with the correct group and flags, we
must _always_ set these. Then we can check to see if there are values
which need migrating, and if not, can safely exit and leave the values
as default.
2020-06-30 13:54:01 +02:00
Kevin Ansfield
fa2c9a4da1 Added missing PUBLIC flag for logo setting in migration
refs #10318

- `logo` setting should have had the `PUBLIC` flag but it was missed
2020-06-30 12:21:22 +01:00
Fabien O'Carroll
0a257c4b5d Updated settings api to ignore stripe_connect settings
refs #10318

These settings are only allowed to be set via the stripe_connect_integration_token key
2020-06-30 11:27:43 +02:00
Fabien O'Carroll
3b8cdb1657 Fixed migrations for missing settings
refs #10318

If migrating from a previous version that does not include the setting
being migrated from we can safely not update the new setting, and just
rely on its default value being present. When rolling back we can use
defaults if the new setting does not exist.
2020-06-30 10:33:55 +02:00
Fabien O'Carroll
bf736717e4 Fixed members default-settings default values
refs #10318
2020-06-29 19:34:14 +02:00
Hannah Wolfe
cd3f8e9e2d Reordered routes to load taxonomies last
closes #11936

- the driving reason for doing this is so that when posts and taxonomies all have the same route,
  e.g. /:slug/ the /edit route won't default to looking up tags
  this is the issue outlined in #11936
- the side effect of this is that in this case, all routes will default to looking up posts before tags,
  so if you have a /test/ post and a /test/ tag, prior to this change, the tag would have been loaded first
  but posts are more important than tags, and should have higher priority, so this change is considered correct
2020-06-29 17:45:47 +01:00
Rish
4ccfca8c00 Added migration to add types for settings
refs https://github.com/TryGhost/Ghost/issues/10318

- Updates the type field for a setting to specify its type explicitly
- Type can be one of (string, number, boolean, array)
- `object` is allowed a temp type till all the object settings are removed
- existing `type` value for setting was misleading and referring to `group`
- we moved the current `type` value for a setting to `group` here - 4a9e57c170
2020-06-29 15:05:38 +00:00
Fabien O'Carroll
21e9fd93b8 Fixed stripe_plans default settings
refs #10318

These should be fractional amounts so they needed to be 100x larger
2020-06-29 16:44:50 +02:00
Daniel Lockyer
dc131cfff1 Merged 3.21.1 into master
v3.21.1

* tag '3.21.1':
  v3.21.1
  Updated Ghost-Admin to v3.21.1
  Un-skipped populateImageSizes subdir test
  Fixed unit tests interdependency issue
  Fixed `populateImageSizes` handling of images in subdir setups
  🐛 Fixed potentially squashed images in image cards
  Refactored site url data attribute for new members script
2020-06-29 15:30:42 +01:00
Fabien 'egg' O'Carroll
8455d24894
Added migrations for members & stripe setting changes (#11961)
refs #10318

* Added migration for stripe_connect_integration
* Added migration for members_subscription_settings
2020-06-29 16:23:38 +02:00
Fabien 'egg' O'Carroll
ee786aaa5d
Cleaned up members & stripe settings (#11957)
* Updated members default settings

ref #10318

This pulls out the members_subscription_settings & stripe_connect_intgration settings into separate keys

* Updated usage of members_from_address

* Updated stripe_connect usage

* Updated members config to use new settings

* Updated members middleware to use isStripeConnected

* Updated members service to reload correctly

We reload the members-api instance when the related settings change, so
this makes sure we're listening to the correct settings changes

* Updated ghost_head helper to use new settings

* Updated theme middleware to use new settings

* Renamed members_allow_signup -> members_allow_free_signup

* Fixed tests after settings refactor

* Removed  from direct key settings key

* Fixed regression tests for settings api
2020-06-29 16:22:42 +02:00
Fabien 'egg' O'Carroll
3aa68b95e7
Updated migration to assume new settings exist (#11963)
refs #10318

As populateDefaults is run _before_ migrations, the new settings will
already be inserted in the database, so we just need to update their
values and then delete the old settings.
2020-06-29 15:31:16 +02:00
Kevin Ansfield
f45198245f Updated Settings.populateDefaults() to account for available columns
refs https://github.com/TryGhost/Ghost/issues/10318

`Settings.populateDefaults()` is run before migrations during Ghost's startup. This can cause problems when new settings table columns are added (and populated in `default-settings.json`) because `populateDefaults()` was using the model layer which assumes that those columns are available, resulting in `ER_BAD_FIELD_ERROR: Unknown column` type errors.

- query the database for the available `settings` table columns
- switch to using raw knex queries without Bookshelf for insertions so that we're in control of the columns that are added
- use `_.pick` to skip any properties in `default-settings.json` that do not match to an available column - those columns will be added and populated by later migrations
- moving away from using the model to insert settings has the side-effect of not emitting `settings.added/edited` and `settings.x.added/edited` events, this should be fine because `populateDefaults()` is called before anything else is set up and listening
- added a call to `populateDefaults()` in our knex-migrator "before migration" hook so that we have consistent db state across both startup initialised migrations and manually triggered knex migrations
2020-06-29 13:22:10 +00:00
Daniel Lockyer
f45effe363 Updated Ghost-Admin to v3.21.1 2020-06-29 13:25:33 +01:00
Nazar Gargol
097ff9fba9 Fixed grouping error in members CSV importer
no issue

- Import error grouping wasn't working correctly when error didn't have defined context property
- The copy will be refined in final design review
2020-06-30 00:22:52 +12:00
Kevin Ansfield
771908d04b Fixed populateImageSizes handling of images in subdir setups
no issue

- when using subdirectories, images can be stored in the database both with and without the subdirectory prefix. We weren't taking that into account and so images without the subdirectory were not having the `/content/images/` prefix removed when passed to the storage adapter resulting in the storage adapter not finding the image
2020-06-29 11:19:56 +01:00
Rish
14040747eb Refactored site url data attribute for new members script
refs https://github.com/TryGhost/members.js/issues/54

- Previously new meta tag was added which defined the site url which was used by new script
- Removes meta tag and instead directly uses a data attribute defined on script tag which it can read on load, making it much simpler
2020-06-29 11:19:28 +01:00
Fabien O'Carroll
79c3709fb9 Updated ALL permissions migrations to use the utils
no-issue
2020-06-25 19:15:25 +02:00
Fabien O'Carroll
ec119c458c Added migration utils for permissions
no-issue

These utils are the first steps toward getting the models out of our
migrations! The utils here interact directly with the database and where
possible don't reinvent the wheel, by using smaller building blocks to
build more comples ones.
2020-06-25 19:15:25 +02:00
Nazar Gargol
91e3630f36 Updated settings API v3 tests to check for correct types returned for specific keys
refs https://github.com/TryGhost/Ghost/issues/10318
refs 476fca6e5b

- Symetric change to one done in referenced commit which maps fields for API v2
2020-06-25 17:36:10 +12:00
Nazar Gargol
e394b5ad9c Added naive settings type options parameter support to settings API v2
refs TryGhost/Ghost#10318
refs 8fc526ff6

- This is symetric change to one done for v3 API (commited as 8fc526ff6)
- Added 'core' filtering for v2 API controller
2020-06-25 16:32:16 +12:00
Kevin Ansfield
05ac53ffa1 Fixed defaultValue for sharedViews setting
refs https://github.com/TryGhost/Ghost/issues/10318

- `shared_views` is always an array (Ghost-Admin has handling to revert it to an array if it's anything else) but it's default value was set to an empty object
2020-06-24 15:15:55 +01:00
Kevin Ansfield
e9b6e8eca7 Removed API filter of settings now correctly grouped in "core"
refs https://github.com/TryGhost/Ghost/issues/10318

- `members_public_key` and `members_private_key` are now correctly grouped under `core` which is already filtered out so we don't need the extra filter to exclude them
2020-06-24 14:55:50 +01:00
Nazar Gargol
8fc526ff6e Added back compatible support for renamed settings types
refs https://github.com/TryGhost/Ghost/issues/10318
refs 1dc0405803

- Adds 1:1 mapping for filtering options  to renamed settings "type" to "gorup"
- Ignores the name changes and any old types
- Detailsed type -> group mappings can be checked in the refereneced migration commit
2020-06-25 00:55:40 +12:00
Kevin Ansfield
1dc0405803
Added migration to populate settings.{group,flags} (#11954)
refs https://github.com/TryGhost/Ghost/issues/10318

- maps old `settings.type` values to new `settings.type/group` values
  - uses an explicit map so that we don't lose information and can safely roll back even though we're modifying `settings.type` too
  - updates `settings.type` values too to keep code working while we switch to using `settings.group`
- sets the `settings.group` value for all settings which are keeping the same group as their current type
- adapts `settings.type` validations to match new groups
- adds flags to specific settings, both in the migration for existing settings records and in default-settings.json for new settings records
2020-06-24 12:38:18 +01:00
Kevin Ansfield
4a9e57c170
Added migration to add settings.{group,flags} columns (#11951)
refs https://github.com/TryGhost/Ghost/issues/10318

- `group`
  - to replace the `type` column, provides a more descriptive name for the columns use
  - for existing sites it will be populated by migrating data from the `type` column in a later migration
  - for new sites a minimal update has been added to `parseDefaultSettings()` to populate the `group` field when settings are created during startup - fixes the NOT NULL constraint on `settings.group`
- `flags`
  - signifies special handling that is different to other settings in a group
  - eg, `PUBLIC,RO` would indicate that the setting is available via unauthenticated endpoints and is read-only
2020-06-24 11:58:15 +01:00
Nazar Gargol
118b7033e0 Fixed input serializer for setting's PUT methods in Admin APIs
refs https://github.com/TryGhost/Ghost/issues/10318

- There was a copy/paste error and we didn't have a test to pick it up. Will follow up with a regression test to make sure it doesn't happen again
2020-06-24 22:11:03 +12:00
Nazar Gargol
8ea245fc7c Fixed importer mapping for renamed default_locale, active_timezone, ghost_* settings
refs #10318
refs 2614565d5a

- Adds importer mapping for fields as in referenced migration
- The intention is to allow exports from v2/v3 to still be compatilbe with current version
- Adds a mapper for deprecated fields and imports them with new settings "keys"
2020-06-24 14:19:52 +12:00
Nazar Gargol
2de1c07dd4 Renamed settings key brand.primaryColor to accent_color
refs #10318
2614565d5a

- Changes naming throughout the codebase matching the change in the
referenced commit
2020-06-24 14:18:13 +12:00
Nazar Gargol
f0a811e9fe Renamed ghost_head/ghost_foot to codeinjection_head/codeinjection_foot
refs #10318
refs 2614565d5a

- Renamed ghost_head/ghost_foot in settings to match the new names
introduced in migrations
- Above change lead to reshufling in the mappings in input/output
serializers
- Makes sure change is compatible with v2 API
2020-06-24 14:18:13 +12:00
Nazar Gargol
5f1060a8bf Renamed settings keys active_timezone to timezone
refs https://github.com/TryGhost/Ghost/issues/10318
refs 2614565d5a

- Renames to match referenced migration renames
- Fixed API responses so they are consistent with newly renamed fields
- Not returning lang and timezone keys from settings in API v2 ther rest should be returned in API v3/canary
2020-06-24 14:11:20 +12:00
Nazar Gargol
c4dff2d50e Renamed settings keys from default_locale to lang
refs #10318
refs 2614565d5a

- Renames default_locale settings key as introduced in referenced
migration
2020-06-24 14:07:24 +12:00
Kevin Ansfield
8a817050c5
Organised default settings into new groups (#11952)
refs https://github.com/TryGhost/Ghost/issues/10318

- precursor to migrating from `settings.type` to `settings.group`
- renames `blog` type to `site`
- renames `bulk_email` type to `email`
- moves settings out of `site` (previously `blog`) into more appropriate groups such as `core` or individual feature groups
2020-06-23 20:49:08 +01:00
Rish
adb5adea57 Fixed incorrect brand structure in settings migration
no issue

- Previous migration in commit - 2614565d5a - was using incorrect json structure for brand -> primary_color
- Fixes the up/down migrations to work on correct existing `brand: primary_color` structure
2020-06-23 22:13:32 +05:30
Naz
2614565d5a
Settings keys renames (#11948)
refs https://github.com/TryGhost/Ghost/issues/10318

- Adds following renames to settings table keys:
  'default_locale' -> 'lang'
  'active_timezone' -> 'timezone'
  'ghost_head' -> 'codeinjection_head'
  'ghost_foot' -> 'codeinjection_foot'
  'brand.publicationColor' -> 'accent_color'
- The renames are done to match revised naming conventions and naming
exposed through APIs
- Supersedes this revert - 1eeb5a60b8 and #11946
2020-06-24 01:10:25 +12:00
Nazar Gargol
5c800a6fe7 Revert "Revert "Added migration to remove deprecated values from the settings table (#11942)""
This reverts commit 32cb9c7f94.

Emergency release has been canceled changes can stay in master
2020-06-24 01:07:29 +12:00
Kevin Ansfield
db66f1cbbd Fixed populateImageSizes handling of images in subdir setups
no issue

- when using subdirectories, images can be stored in the database both with and without the subdirectory prefix. We weren't taking that into account and so images without the subdirectory were not having the `/content/images/` prefix removed when passed to the storage adapter resulting in the storage adapter not finding the image
2020-06-23 12:47:20 +01:00
Nazar Gargol
32cb9c7f94 Revert "Added migration to remove deprecated values from the settings table (#11942)"
This reverts commit bdd8049e06.

- Revert was done to allow emergency releaese
2020-06-23 19:29:01 +12:00
Nazar Gargol
1eeb5a60b8 Revert "Renamed settings table keys (#11946)"
This reverts commit d8cdeb6d01.

- Revert was done to allow emergency releaese
2020-06-23 19:27:52 +12:00
Naz
d8cdeb6d01
Renamed settings table keys (#11946)
refs https://github.com/TryGhost/Ghost/issues/10318

- Adds following renames to settings table keys:
  'default_locale' -> 'lang'
  'active_timezone' -> 'timezone'
  'ghost_head' -> 'codeinjection_head'
  'ghost_foot' -> 'codeinjection_foot'
  'brand.publicationColor' -> 'accent_color'
- The renames are done to match revised naming conventions and naming
exposed through APIs
2020-06-23 19:09:12 +12:00
Naz
bdd8049e06
Added migration to remove deprecated values from the settings table (#11942)
refs https://github.com/TryGhost/Ghost/issues/10318
refs https://github.com/TryGhost/Ghost/pull/11939

- Removes force_i18n, permalinks, and members_session_secret settings
values as their respective use has been removed from the codebase.
- Related code changes in referenced PR
2020-06-23 19:07:33 +12:00
Nazar Gargol
b78dbfbbc2 Removed use of deprecated setting flags from the codebase
refs #10318
refs https://github.com/TryGhost/Ghost/pull/11942

- Removes force_i18n, permalinks, and members_session_secret usage from the codebase
-  We deprecated these flasgs and have not used since Ghost v2. It's good time to remove them before we introduce bigger changes to how `settings` table opeartes.
- Fixed importer test. The test was meant to check if string values were converted properly, the check agains boolean didn't make much sense in this context, so removed it.
- Following this change are going to come ralated migrations to fix existing data (see ref)
2020-06-23 11:58:19 +12:00
Rish
a86878ab8f Refactored site url data attribute for new members script
refs https://github.com/TryGhost/members.js/issues/54

- Previously new meta tag was added which defined the site url which was used by new script
- Removes meta tag and instead directly uses a data attribute defined on script tag which it can read on load, making it much simpler
2020-06-22 17:30:33 +05:30
Daniel Lockyer
1b13468fe5 Updated Ghost-Admin to v3.21.0 2020-06-22 11:33:58 +01:00
Rish
77e1ada6c6 Added new settings for members modal
no issue

-  reverts commit 87c31444fd but with modifications to settings naming
-  Adds new settings for members modal customization to default settings
- `portal_button` controls the visibility of beacon in members modal
- `portal_name` controls the visibility of name field in signup
- `portal_plans` controls the visibility of plans allowed for member to signup with
-  Adds stripe connect check to determine if stripe is setup or not
-  Adds the 3 new settings to members site data
-  Updates to snake case naming for members site API data
2020-06-19 21:35:49 +05:30
Nazar Gargol
93e8ee83d9 Extracted members CSV related code into @tryghost/members-csv package
no issue

- Moves out CSV parsing and serialization related code into separate package as a part of push to modularize Ghost repo.
- Next up is to remove `csv-parser` dependency from this new package
2020-06-19 19:55:47 +12:00
Nazar Gargol
fabe06c5c5 Colocated readCSV util with members input serializer
refs 5a6ce5abfe

- Allows to move files and keep the history in one go.
- 'csv-parser' will be upgraded to 'papaparse' lib in a new package
2020-06-19 18:24:51 +12:00
Nazar Gargol
5a6ce5abfe Extracted members csv input serialization logic into separate module
refs a6b5a82f09

- This is prep work for extraction into members repository.
2020-06-19 17:58:33 +12:00
Nazar Gargol
a6b5a82f09 Extracted members csv serialization logic into separate module
no issue

- This is prep work for extraction into members repository.
2020-06-19 17:23:05 +12:00
Nazar Gargol
e698f9f95c Fixed handling for grouped validation errors during members import
refs 7904c303a7

- Model's validation errors are sometimes returned as an array of errors, in those cases they should be flattened to before returning to the client. This way there is more insight into what went wrong during the import.
2020-06-19 16:34:23 +12:00
Fabien O'Carroll
4716d13d13 Added API endpoints for disconnecting Stripe
no-issue
pr: https://github.com/TryGhost/Ghost/pull/11930

- Upgraded @tryghost/members-api to 0.23.0

  This version includes a new method hasActiveStripeSubscriptions

- Added /admin/members/hasActiveStripeSubscriptions

  This can be used to determine whether or not we should allow removing
  the stripe keys.

- Added /admin/settings/stripe/connect

  This can be used to delete a Stripe Connect integration, provided
  there are not active subscriptions
2020-06-18 18:42:20 +02:00
Daniel Lockyer
ef7555f9aa Updated Ghost-Admin to v3.20.3 2020-06-18 16:20:13 +01:00
Kevin Ansfield
2553da1bee Fixed mobiledoc.populateImageSizes handling of empty image card payloads
no issue

- it's possible that image cards can be saved when they have no `src` attribute (placeholders)
2020-06-18 15:32:42 +01:00
Daniel Lockyer
3070bfa486 Updated Ghost-Admin to v3.20.2 2020-06-18 15:10:11 +01:00
Rish
87c31444fd Revert "Added new settings for members modal settings (#11927)"
This reverts commit ea7d450a9b.
2020-06-18 19:06:13 +05:30
Kevin Ansfield
4fb39f29cc Fixed population of image sizes when forcing re-render of posts
no issue

- fixed incorrect method name when calling
- fixed problem with setting post.mobiledoc to a promise rather than waiting for the size population to finish and setting it to a mobiledoc string
2020-06-18 14:02:53 +01:00
Kevin Ansfield
95525eeadc Allowed ?formats param in Admin API post+page create/edit requests
no issue

- `?formats` is useful when you want to get `html` back from the Admin API when creating/editing posts, otherwise you're forced to make a following `GET` request
2020-06-18 13:59:01 +01:00
Rishabh Garg
ea7d450a9b
Added new settings for members modal settings (#11927)
no issue

-  Adds new settings for members modal customization to default settings
- `membersjs_show_beacon` controls the visibility of beacon in members modal
- `membersjs_show_signup_name` controls the visibility of name field in signup
- `membersjs_allowed_plans` controls the visibility of plans allowed for member to signup with
-  Adds stripe connect check to determine if stripe is setup or not
-  Adds the 3 new settings to members site data
2020-06-18 17:11:08 +05:30
Daniel Lockyer
4a2114e405 Updated Ghost-Admin to v3.20.1 2020-06-17 17:13:05 +01:00
Kevin Ansfield
f27282bc3f
Added mobiledoc transform to populate missing image sizes on forced re-render (#11924)
no issue

- adds `populateImageSizes()` to our mobiledoc lib module
  - uses `image-size` lib to speed up reading of image dimensions
  - for local images, use storage adapter with same guards as used by `handle-image-sizes` middleware so that we don't insert srcsets for images that aren't transformable
  - for unsplash images, remove any width and crop params from the url so it points to the full-size image
- use `populateImageSizes(mobiledoc)` to modify post model's mobiledoc when re-rendering
2020-06-17 13:12:32 +01:00
Kevin Ansfield
4537ccd329 🐛 Fixed embed/bookmark retrieval for sites that require cookies during redirects
no issue

- known failing case was Medium. Some *.medium.com articles triggered redirects but without cookies being stored/sent across the redirect requests it would cause an infinite redirect loop and we'd abort after 10 redirects
- use `got`'s [cookie support](https://github.com/sindresorhus/got/tree/v9.6.0#cookies) via `tough-cookie` so that we can create bookmark cards for medium.com and other sites with similar problems
2020-06-17 10:03:36 +01:00
Kevin Ansfield
86f14c28a1 Fixed no-shadow linter warnings in image-size.js 2020-06-16 12:36:55 +01:00
Daniel Lockyer
dc1997fc89 Updated Ghost-Admin to v3.20.0 2020-06-16 11:55:57 +01:00
Hannah Wolfe
a9759736d6
🐛 Fixed private blogging exposing 404 and robots (#11922)
- There were various cases where it was possible to trigger a private site to display a 404 instead of redirecting to /private/
- Private mode was also not always displaying the correct robots.txt
- This PR includes tests for all cases in test/frontend-acceptance/default_routes_spec.js & where possible the unit tests have also been updated for completeness
- Fixing the 404 issues required
    - Better handling of paths using req.path instead of req.url in filterPrivateRoutes
    - Additional error handling, to cover the case that a tag/author RSS feed does not exist
 - Fixing the robots.txt required the order of middleware to be changed, so that private blogging gets a chance to render first
    - NOTE private blogging is the only app with a setupMiddleware function so nothing else is affected
2020-06-16 11:42:32 +01:00
Fabien O'Carroll
c5b8dab523 🐛 Fixed custom redirects for subdirectory setups
no-issue

This issue only occurs when using custom redirects with a subdirectory
setup, and the path to be redirected from is expressed as a regex, and
the url that is being redirected to is not an external url.

The issue has a few components:

- Redirect paths as a regex generally use the ^ to ensure that they
  match the beginning of the path.

- The path that the regex is matched against conditionally excludes the
  subdirectory, specifically, the subdirectory is excluded for external
  urls

These combined means you end up with a regex like /^\/custom-redirect/
and a path like /subdir/custom-redirect, these will not match/replace
correctly, and you'll end in an infinite redirect loop.

The fix here is to *always* remove the subdirectory when testing regex's
and then conditionally adding it back *only* for the redirect, and only
if it is an internal redirect
2020-06-16 10:24:13 +02:00
Nazar Gargol
5f00619d1a Renamed members import/export endpoints
refs cb26fd9305

- Reasoning behind the rename in referenced commit
2020-06-16 18:08:50 +12:00
Hannah Wolfe
f4641aabe4 Rename private blogging mw functions
- This is just a nicety, trying to make it easier to follow the logic of private blogging
2020-06-15 20:55:59 +01:00
Hannah Wolfe
4ae907781a Private blogging misc cleanup
This is a bunch of small changes, that simplifies working with the private blogging module:
- remove reference to really old paginated RSS behaviour
- remove handling for /rss and allow our standard redirects to redirect to /rss/ and then execute
- readd should to tests so that they can be run standalone
- fix eslint warning
2020-06-15 20:16:13 +01:00
Kevin Ansfield
e0d1763424 🐛 Fixed fallback to bookmark card when pasting localhost Ghost url
no issue

- localhost check was causing an immediate "no provider" error when fetching oembed for a localhost url and bypassing the bookmark fallback even when the configured site lives on localhost
- allow `localhost:port` through in oembed endpoints when it matches the configured url
2020-06-15 18:38:43 +01:00
Kevin Ansfield
ce8bc9d4bf Added imageOptimization:srcsets config to allow disabling of srcset attrs in content
no issue

- pass `srcsets` value through to mobiledoc renderer
  - it will stop adding `srcset` attributes to images when explicitly set to `false`
- adds `reload()` method to `mobiledocLib` so that memoized instances of the renderer can be reset during tests that change config values
2020-06-15 16:45:36 +01:00
Kevin Ansfield
73429589bd Reverted to previous user-agent for bookmark and oembed requests
no issue

- need to document changes so that external systems can adapt to the revised user-agent
2020-06-15 16:02:04 +01:00
Hannah Wolfe
27066ce910
🐛 Fixed missing text version in bulk email (#11919)
closes #11917

- Pass text-only version to mailgun as `text` not `plaintext`
- This ensures we send a text-only version of the email, and this in turn should help to improve spam scores
2020-06-15 15:31:09 +01:00
Rishabh Garg
84d0a46c6e
Removed "from address" overwrite in importer (#11914)
refs https://github.com/TryGhost/Ghost/issues/11414

- Importing data currently overwrites the existing "from address" with new value
- "from address" needs to go through email validation flow before update which was bypassed
- Updates importer to not allow overwrite for "from address" and use existing
- Adds test for "from address" overwrite
2020-06-15 12:22:55 +05:30
Hannah Wolfe
edff7ac853
Adding INR currency support (#11911)
- We have many customers asking for INR as there are special rules in Stripe for this currency
- As well as a desire for local-selling
- Meaning it's not valid to use e.g. USD instead
2020-06-12 19:36:25 +01:00
Kevin Ansfield
424e43975c Allowed ?force_reload=true through on pages edit endpoint
refs 008f86fc29

- same as posts. Related functionality to handle the query param exists on the Post model
2020-06-12 19:15:19 +01:00
Fabien O'Carroll
63f1e57036 Fixed Members Stripe config for Stripe Connect
no-issue

This was initially missed as local settings always had the original
stripeDirect keys, this ensures that regardless of Connect vs Direct vs
Both vs Neither that the config is correct.

Also ensures that the Members API instance is reloaded when the Stripe
Connect settings are changed.
2020-06-12 19:22:36 +02:00
Kevin Ansfield
008f86fc29 Added ?force_rerender=true param to posts edit endpoint
no issue

- there are various situations where we adapt/fix/improve our mobiledoc->html output over time but we didn't have a way of updating old content without manually editing the mobiledoc and saving, or running an expensive migration to re-render all old content
- this adds a `?force_rerender=true` query param to the `PUT .../admin/posts/:id/` endpoint that allows the `html` field to be re-generated without modifying the `mobiledoc` field contents
2020-06-12 18:05:57 +01:00
Kevin Ansfield
5d0d0b397e Fixed no-shadow linting warnings in models/post.js 2020-06-12 17:55:40 +01:00
Nazar Gargol
cb26fd9305 Changed members upload validation endpoint to /members/upload/validate
no issue

- The intention is to move away from using file type names in URLs. This endpoint is meant to correspond to current `POST /members/csv` endpoint, that is  planned to be renamed into `/members/upload`. And the `GET /members/csv` to be renamed to  `/members/download` respectively.
2020-06-12 23:31:56 +12:00