Commit Graph

8019 Commits

Author SHA1 Message Date
Fabien O'Carroll
f3b4a538af Updated the validateSettings method to check stripe_plans
refs #11765

This ensures that plans will always be set to 1 of the currency
2020-07-03 10:37:13 +02:00
Fabien O'Carroll
38fce4efc5 Cleaned up data validation module
no-issue

Removes overly verbose repetition of identifiers making this easier to read
2020-07-03 10:37:13 +02:00
Fabien O'Carroll
774187ced5 Cleaned up onValidate method on settings model
no-issue

Explicit arguments and async/await make this nicer on the eyes
2020-07-03 10:37:13 +02:00
Kevin Ansfield
e9ae0d9baa 🐛 Fixed srcset and sizes attributes being rendered when image resizing unavailable
closes https://github.com/TryGhost/Ghost/issues/11944

- updates `@tryghost/image-transform` to version that exposes `canTransformFiles()` which checks for `sharp` availibility
- updates `@tryghost/kg-default-cards` to version that accepts a `canTransformImage()` method as an option
- updates our `mobiledoc` lib to pass a `canTransformImage()` function that returns false if sharp is unavailable, the image extension is not supported, or the storage engine in use does not support image transforms
- updates `populateImageSizes` to fetch image sizes when transforms are unavailable as the render/not-render is now handled in the renderer and we don't need to worry about adding size information to the mobiledoc source
2020-07-02 18:03:22 +01:00
Nazar Gargol
6902f86b40 Added field mapping support for members import CSVs
no issue

- This is a precursor work allowing client to send csv to JSON
field mapping hash as a `mapping` field parameter in the incoming
request.
2020-07-02 15:18:39 +12:00
Daniel Lockyer
27e49a4edc Updated Ghost-Admin to v3.22.1 2020-07-01 18:38:32 +01:00
Kevin Ansfield
9d7665bb43 🐛 Fixed global code injection not being output
refs https://github.com/TryGhost/Ghost/issues/10318

- settings key rename was missed in the `ghost_head` and `ghost_foot` helpers
2020-07-01 17:44:59 +01:00
Kevin Ansfield
fd4a011995
🐛 Fixed settings cache being out of sync after migrations (#11987)
refs https://github.com/TryGhost/Ghost/issues/10318

- re-initialize settings cache after migrations by shutting down to clean up event listeners then and calling `init` again
- important to ensure `db.ready` event is not emitted until settings have finished re-initializing to avoid problems with background processes using the db connection which is disconnected/re-connected or being kicked off with out-of-date settings
2020-07-01 17:16:57 +01:00
Fabien 'egg' O'Carroll
cbdf33bfb7
Fixed @price.currency_symbol helper
closes #11984

The refactor to new settings lost the uppercasing of the currency string, this puts it back
2020-07-01 13:26:30 +02:00
Daniel Lockyer
e457f2018c Updated Ghost-Admin to v3.22.0 2020-07-01 10:49:49 +01:00
Nazar Gargol
d03f674a2f Fixed failing members import tests
refs 05f6faf846

- The logic in "Stripe" error detection depended on error's message string matching, which is not a good practice in general. Had fixed it do to exact match on "context" of the error we throw internally and left more extensive comment about why things are implemented the way they are.
2020-07-01 19:03:12 +12:00
Peter Zimon
05f6faf846 Updated error messages for member import
no refs.
- refined errors for member import post-validation
2020-07-01 07:23:20 +02:00
Fabien O'Carroll
44c03eb0f4 Fixed migration exiting for single missing setting
refs #10318

`return` would break the whole loop, `continue` will continue to the
next iteration
2020-06-30 18:38:55 +02:00
Fabien O'Carroll
0074324b10 Fixed migration when brand setting is null
refs #10318

This would throw if the setting was null, we catch and use the default
instead.
2020-06-30 18:30:05 +02:00
Kevin Ansfield
9810c41dca 🐛 Fixed facebook/twitter images having relative URLS in API output
no issue

Output serializer's url util was expecting `og_image` and `twitter_image` to be top-level attributes in the `attrs` object but they are actually nested under `posts_meta`.

- updated the code to use lodash's `get/set()` so that we can work with paths for easier handling of nested objects
- fixed unit tests where the mocked data under test did not match real-world data
2020-06-30 16:40:34 +01:00
Jonas Fabisiak
58ac033b97
Updated head to only output custom favicons (#11949)
closes #11932

- as per the issue, there is no need to output a <link> tag if the favicon is the default /favicon.ico, as all browsers automatically check for this 
- instead the favicon <link> is only output if a custom favicon has been set, telling the browser to look somewhere different to its default location
- some of the tests expected 3 links in the html head, but through the favicon change there are often only 2.

Co-authored-by: RenCloud <rencloud@pop-os.localdomain>
2020-06-30 15:07:08 +01:00
Hannah Wolfe
fa91c6c954
🐛 Moved access to an API property (#11967)
closes #11574

- the current implementation of the access property has it frontend only, and wired up only in one place
- this leaves it only available in a handful of places, e.g. can't use it in a post loop or get helper
- the current implementation also fails logically if the html content of the post is blank

This fix moves the behaviour to the API

- this ensures the field is always available no matter what context you are in
- it also updates the logic to use the same membersHasAccess logic as is used to gate the post, so it's always correct

TODO: should reconsider the location of this code
2020-06-30 14:46:46 +01:00
Rish
212e39f772 Updated settings importer to handle new mapping
refs https://github.com/TryGhost/Ghost/issues/10318

- Skipped `members_subscription_settings` and `stripe_connect_integration` from settings import
- Fixed import data field filtering for core and theme groups
2020-06-30 17:38:36 +05:30
Rish
d5f68dbbc5 Updated serializers/model layer validation using settings type
refs https://github.com/TryGhost/Ghost/issues/10318

- Updates `boolean` serialization in v2/canary serializers to apply only for `boolean` type settings
- Updates `boolean` transformation in model layer `format`/`parse` to check on `boolean` type setting
- Removes error thrown on Read-only setting for settings edit endpoint
- Updates v2/canary input serializers to remove any Read-only settings (using RO flag) to avoid edits
- Added type/group mappings in the importer when pre-migration settings table import data is present
- Updates tests
2020-06-30 17:38:36 +05:30
Rish
8a50a3e9c9 Updated default settings and schema with updated types
refs https://github.com/TryGhost/Ghost/issues/10318

- Updates default settings to contain correct type and validation for each setting
- Updates `populateDefaults` to correctly insert type value for new settings
- Updates settings schema to allow only select types - `array`, `number`, `boolean`, `string`
- `object` is a temporary type allowed till we get rid of all JSON object settings
2020-06-30 17:38:36 +05:30
Fabien O'Carroll
01633470f8 Switched on Stripe Connect by default
no-issue

This flags switches off the old API Key UI and replaces it with the new
Stripe Connect flow!
2020-06-30 14:07:33 +02:00
Fabien O'Carroll
a5751341b6 Added RO flag to members_from_address
refs #10318

This was incorrectly missing before
2020-06-30 13:54:01 +02:00
Fabien O'Carroll
f2f40a5f4a Updated migrations to always set group,types,flags
refs #10318

Because settings are not populated with the correct group and flags, we
must _always_ set these. Then we can check to see if there are values
which need migrating, and if not, can safely exit and leave the values
as default.
2020-06-30 13:54:01 +02:00
Kevin Ansfield
fa2c9a4da1 Added missing PUBLIC flag for logo setting in migration
refs #10318

- `logo` setting should have had the `PUBLIC` flag but it was missed
2020-06-30 12:21:22 +01:00
Fabien O'Carroll
0a257c4b5d Updated settings api to ignore stripe_connect settings
refs #10318

These settings are only allowed to be set via the stripe_connect_integration_token key
2020-06-30 11:27:43 +02:00
Fabien O'Carroll
3b8cdb1657 Fixed migrations for missing settings
refs #10318

If migrating from a previous version that does not include the setting
being migrated from we can safely not update the new setting, and just
rely on its default value being present. When rolling back we can use
defaults if the new setting does not exist.
2020-06-30 10:33:55 +02:00
Fabien O'Carroll
bf736717e4 Fixed members default-settings default values
refs #10318
2020-06-29 19:34:14 +02:00
Hannah Wolfe
cd3f8e9e2d Reordered routes to load taxonomies last
closes #11936

- the driving reason for doing this is so that when posts and taxonomies all have the same route,
  e.g. /:slug/ the /edit route won't default to looking up tags
  this is the issue outlined in #11936
- the side effect of this is that in this case, all routes will default to looking up posts before tags,
  so if you have a /test/ post and a /test/ tag, prior to this change, the tag would have been loaded first
  but posts are more important than tags, and should have higher priority, so this change is considered correct
2020-06-29 17:45:47 +01:00
Rish
4ccfca8c00 Added migration to add types for settings
refs https://github.com/TryGhost/Ghost/issues/10318

- Updates the type field for a setting to specify its type explicitly
- Type can be one of (string, number, boolean, array)
- `object` is allowed a temp type till all the object settings are removed
- existing `type` value for setting was misleading and referring to `group`
- we moved the current `type` value for a setting to `group` here - 4a9e57c170
2020-06-29 15:05:38 +00:00
Fabien O'Carroll
21e9fd93b8 Fixed stripe_plans default settings
refs #10318

These should be fractional amounts so they needed to be 100x larger
2020-06-29 16:44:50 +02:00
Daniel Lockyer
dc131cfff1 Merged 3.21.1 into master
v3.21.1

* tag '3.21.1':
  v3.21.1
  Updated Ghost-Admin to v3.21.1
  Un-skipped populateImageSizes subdir test
  Fixed unit tests interdependency issue
  Fixed `populateImageSizes` handling of images in subdir setups
  🐛 Fixed potentially squashed images in image cards
  Refactored site url data attribute for new members script
2020-06-29 15:30:42 +01:00
Fabien 'egg' O'Carroll
8455d24894
Added migrations for members & stripe setting changes (#11961)
refs #10318

* Added migration for stripe_connect_integration
* Added migration for members_subscription_settings
2020-06-29 16:23:38 +02:00
Fabien 'egg' O'Carroll
ee786aaa5d
Cleaned up members & stripe settings (#11957)
* Updated members default settings

ref #10318

This pulls out the members_subscription_settings & stripe_connect_intgration settings into separate keys

* Updated usage of members_from_address

* Updated stripe_connect usage

* Updated members config to use new settings

* Updated members middleware to use isStripeConnected

* Updated members service to reload correctly

We reload the members-api instance when the related settings change, so
this makes sure we're listening to the correct settings changes

* Updated ghost_head helper to use new settings

* Updated theme middleware to use new settings

* Renamed members_allow_signup -> members_allow_free_signup

* Fixed tests after settings refactor

* Removed  from direct key settings key

* Fixed regression tests for settings api
2020-06-29 16:22:42 +02:00
Fabien 'egg' O'Carroll
3aa68b95e7
Updated migration to assume new settings exist (#11963)
refs #10318

As populateDefaults is run _before_ migrations, the new settings will
already be inserted in the database, so we just need to update their
values and then delete the old settings.
2020-06-29 15:31:16 +02:00
Kevin Ansfield
f45198245f Updated Settings.populateDefaults() to account for available columns
refs https://github.com/TryGhost/Ghost/issues/10318

`Settings.populateDefaults()` is run before migrations during Ghost's startup. This can cause problems when new settings table columns are added (and populated in `default-settings.json`) because `populateDefaults()` was using the model layer which assumes that those columns are available, resulting in `ER_BAD_FIELD_ERROR: Unknown column` type errors.

- query the database for the available `settings` table columns
- switch to using raw knex queries without Bookshelf for insertions so that we're in control of the columns that are added
- use `_.pick` to skip any properties in `default-settings.json` that do not match to an available column - those columns will be added and populated by later migrations
- moving away from using the model to insert settings has the side-effect of not emitting `settings.added/edited` and `settings.x.added/edited` events, this should be fine because `populateDefaults()` is called before anything else is set up and listening
- added a call to `populateDefaults()` in our knex-migrator "before migration" hook so that we have consistent db state across both startup initialised migrations and manually triggered knex migrations
2020-06-29 13:22:10 +00:00
Daniel Lockyer
f45effe363 Updated Ghost-Admin to v3.21.1 2020-06-29 13:25:33 +01:00
Nazar Gargol
097ff9fba9 Fixed grouping error in members CSV importer
no issue

- Import error grouping wasn't working correctly when error didn't have defined context property
- The copy will be refined in final design review
2020-06-30 00:22:52 +12:00
Kevin Ansfield
771908d04b Fixed populateImageSizes handling of images in subdir setups
no issue

- when using subdirectories, images can be stored in the database both with and without the subdirectory prefix. We weren't taking that into account and so images without the subdirectory were not having the `/content/images/` prefix removed when passed to the storage adapter resulting in the storage adapter not finding the image
2020-06-29 11:19:56 +01:00
Rish
14040747eb Refactored site url data attribute for new members script
refs https://github.com/TryGhost/members.js/issues/54

- Previously new meta tag was added which defined the site url which was used by new script
- Removes meta tag and instead directly uses a data attribute defined on script tag which it can read on load, making it much simpler
2020-06-29 11:19:28 +01:00
Fabien O'Carroll
79c3709fb9 Updated ALL permissions migrations to use the utils
no-issue
2020-06-25 19:15:25 +02:00
Fabien O'Carroll
ec119c458c Added migration utils for permissions
no-issue

These utils are the first steps toward getting the models out of our
migrations! The utils here interact directly with the database and where
possible don't reinvent the wheel, by using smaller building blocks to
build more comples ones.
2020-06-25 19:15:25 +02:00
Nazar Gargol
91e3630f36 Updated settings API v3 tests to check for correct types returned for specific keys
refs https://github.com/TryGhost/Ghost/issues/10318
refs 476fca6e5b

- Symetric change to one done in referenced commit which maps fields for API v2
2020-06-25 17:36:10 +12:00
Nazar Gargol
e394b5ad9c Added naive settings type options parameter support to settings API v2
refs TryGhost/Ghost#10318
refs 8fc526ff6

- This is symetric change to one done for v3 API (commited as 8fc526ff6)
- Added 'core' filtering for v2 API controller
2020-06-25 16:32:16 +12:00
Kevin Ansfield
05ac53ffa1 Fixed defaultValue for sharedViews setting
refs https://github.com/TryGhost/Ghost/issues/10318

- `shared_views` is always an array (Ghost-Admin has handling to revert it to an array if it's anything else) but it's default value was set to an empty object
2020-06-24 15:15:55 +01:00
Kevin Ansfield
e9b6e8eca7 Removed API filter of settings now correctly grouped in "core"
refs https://github.com/TryGhost/Ghost/issues/10318

- `members_public_key` and `members_private_key` are now correctly grouped under `core` which is already filtered out so we don't need the extra filter to exclude them
2020-06-24 14:55:50 +01:00
Nazar Gargol
8fc526ff6e Added back compatible support for renamed settings types
refs https://github.com/TryGhost/Ghost/issues/10318
refs 1dc0405803

- Adds 1:1 mapping for filtering options  to renamed settings "type" to "gorup"
- Ignores the name changes and any old types
- Detailsed type -> group mappings can be checked in the refereneced migration commit
2020-06-25 00:55:40 +12:00
Kevin Ansfield
1dc0405803
Added migration to populate settings.{group,flags} (#11954)
refs https://github.com/TryGhost/Ghost/issues/10318

- maps old `settings.type` values to new `settings.type/group` values
  - uses an explicit map so that we don't lose information and can safely roll back even though we're modifying `settings.type` too
  - updates `settings.type` values too to keep code working while we switch to using `settings.group`
- sets the `settings.group` value for all settings which are keeping the same group as their current type
- adapts `settings.type` validations to match new groups
- adds flags to specific settings, both in the migration for existing settings records and in default-settings.json for new settings records
2020-06-24 12:38:18 +01:00
Kevin Ansfield
4a9e57c170
Added migration to add settings.{group,flags} columns (#11951)
refs https://github.com/TryGhost/Ghost/issues/10318

- `group`
  - to replace the `type` column, provides a more descriptive name for the columns use
  - for existing sites it will be populated by migrating data from the `type` column in a later migration
  - for new sites a minimal update has been added to `parseDefaultSettings()` to populate the `group` field when settings are created during startup - fixes the NOT NULL constraint on `settings.group`
- `flags`
  - signifies special handling that is different to other settings in a group
  - eg, `PUBLIC,RO` would indicate that the setting is available via unauthenticated endpoints and is read-only
2020-06-24 11:58:15 +01:00
Nazar Gargol
118b7033e0 Fixed input serializer for setting's PUT methods in Admin APIs
refs https://github.com/TryGhost/Ghost/issues/10318

- There was a copy/paste error and we didn't have a test to pick it up. Will follow up with a regression test to make sure it doesn't happen again
2020-06-24 22:11:03 +12:00
Nazar Gargol
8ea245fc7c Fixed importer mapping for renamed default_locale, active_timezone, ghost_* settings
refs #10318
refs 2614565d5a

- Adds importer mapping for fields as in referenced migration
- The intention is to allow exports from v2/v3 to still be compatilbe with current version
- Adds a mapper for deprecated fields and imports them with new settings "keys"
2020-06-24 14:19:52 +12:00
Nazar Gargol
2de1c07dd4 Renamed settings key brand.primaryColor to accent_color
refs #10318
2614565d5a

- Changes naming throughout the codebase matching the change in the
referenced commit
2020-06-24 14:18:13 +12:00
Nazar Gargol
f0a811e9fe Renamed ghost_head/ghost_foot to codeinjection_head/codeinjection_foot
refs #10318
refs 2614565d5a

- Renamed ghost_head/ghost_foot in settings to match the new names
introduced in migrations
- Above change lead to reshufling in the mappings in input/output
serializers
- Makes sure change is compatible with v2 API
2020-06-24 14:18:13 +12:00
Nazar Gargol
5f1060a8bf Renamed settings keys active_timezone to timezone
refs https://github.com/TryGhost/Ghost/issues/10318
refs 2614565d5a

- Renames to match referenced migration renames
- Fixed API responses so they are consistent with newly renamed fields
- Not returning lang and timezone keys from settings in API v2 ther rest should be returned in API v3/canary
2020-06-24 14:11:20 +12:00
Nazar Gargol
c4dff2d50e Renamed settings keys from default_locale to lang
refs #10318
refs 2614565d5a

- Renames default_locale settings key as introduced in referenced
migration
2020-06-24 14:07:24 +12:00
Kevin Ansfield
8a817050c5
Organised default settings into new groups (#11952)
refs https://github.com/TryGhost/Ghost/issues/10318

- precursor to migrating from `settings.type` to `settings.group`
- renames `blog` type to `site`
- renames `bulk_email` type to `email`
- moves settings out of `site` (previously `blog`) into more appropriate groups such as `core` or individual feature groups
2020-06-23 20:49:08 +01:00
Rish
adb5adea57 Fixed incorrect brand structure in settings migration
no issue

- Previous migration in commit - 2614565d5a - was using incorrect json structure for brand -> primary_color
- Fixes the up/down migrations to work on correct existing `brand: primary_color` structure
2020-06-23 22:13:32 +05:30
Naz
2614565d5a
Settings keys renames (#11948)
refs https://github.com/TryGhost/Ghost/issues/10318

- Adds following renames to settings table keys:
  'default_locale' -> 'lang'
  'active_timezone' -> 'timezone'
  'ghost_head' -> 'codeinjection_head'
  'ghost_foot' -> 'codeinjection_foot'
  'brand.publicationColor' -> 'accent_color'
- The renames are done to match revised naming conventions and naming
exposed through APIs
- Supersedes this revert - 1eeb5a60b8 and #11946
2020-06-24 01:10:25 +12:00
Nazar Gargol
5c800a6fe7 Revert "Revert "Added migration to remove deprecated values from the settings table (#11942)""
This reverts commit 32cb9c7f94.

Emergency release has been canceled changes can stay in master
2020-06-24 01:07:29 +12:00
Kevin Ansfield
db66f1cbbd Fixed populateImageSizes handling of images in subdir setups
no issue

- when using subdirectories, images can be stored in the database both with and without the subdirectory prefix. We weren't taking that into account and so images without the subdirectory were not having the `/content/images/` prefix removed when passed to the storage adapter resulting in the storage adapter not finding the image
2020-06-23 12:47:20 +01:00
Nazar Gargol
32cb9c7f94 Revert "Added migration to remove deprecated values from the settings table (#11942)"
This reverts commit bdd8049e06.

- Revert was done to allow emergency releaese
2020-06-23 19:29:01 +12:00
Nazar Gargol
1eeb5a60b8 Revert "Renamed settings table keys (#11946)"
This reverts commit d8cdeb6d01.

- Revert was done to allow emergency releaese
2020-06-23 19:27:52 +12:00
Naz
d8cdeb6d01
Renamed settings table keys (#11946)
refs https://github.com/TryGhost/Ghost/issues/10318

- Adds following renames to settings table keys:
  'default_locale' -> 'lang'
  'active_timezone' -> 'timezone'
  'ghost_head' -> 'codeinjection_head'
  'ghost_foot' -> 'codeinjection_foot'
  'brand.publicationColor' -> 'accent_color'
- The renames are done to match revised naming conventions and naming
exposed through APIs
2020-06-23 19:09:12 +12:00
Naz
bdd8049e06
Added migration to remove deprecated values from the settings table (#11942)
refs https://github.com/TryGhost/Ghost/issues/10318
refs https://github.com/TryGhost/Ghost/pull/11939

- Removes force_i18n, permalinks, and members_session_secret settings
values as their respective use has been removed from the codebase.
- Related code changes in referenced PR
2020-06-23 19:07:33 +12:00
Nazar Gargol
b78dbfbbc2 Removed use of deprecated setting flags from the codebase
refs #10318
refs https://github.com/TryGhost/Ghost/pull/11942

- Removes force_i18n, permalinks, and members_session_secret usage from the codebase
-  We deprecated these flasgs and have not used since Ghost v2. It's good time to remove them before we introduce bigger changes to how `settings` table opeartes.
- Fixed importer test. The test was meant to check if string values were converted properly, the check agains boolean didn't make much sense in this context, so removed it.
- Following this change are going to come ralated migrations to fix existing data (see ref)
2020-06-23 11:58:19 +12:00
Rish
a86878ab8f Refactored site url data attribute for new members script
refs https://github.com/TryGhost/members.js/issues/54

- Previously new meta tag was added which defined the site url which was used by new script
- Removes meta tag and instead directly uses a data attribute defined on script tag which it can read on load, making it much simpler
2020-06-22 17:30:33 +05:30
Daniel Lockyer
1b13468fe5 Updated Ghost-Admin to v3.21.0 2020-06-22 11:33:58 +01:00
Rish
77e1ada6c6 Added new settings for members modal
no issue

-  reverts commit 87c31444fd but with modifications to settings naming
-  Adds new settings for members modal customization to default settings
- `portal_button` controls the visibility of beacon in members modal
- `portal_name` controls the visibility of name field in signup
- `portal_plans` controls the visibility of plans allowed for member to signup with
-  Adds stripe connect check to determine if stripe is setup or not
-  Adds the 3 new settings to members site data
-  Updates to snake case naming for members site API data
2020-06-19 21:35:49 +05:30
Nazar Gargol
93e8ee83d9 Extracted members CSV related code into @tryghost/members-csv package
no issue

- Moves out CSV parsing and serialization related code into separate package as a part of push to modularize Ghost repo.
- Next up is to remove `csv-parser` dependency from this new package
2020-06-19 19:55:47 +12:00
Nazar Gargol
fabe06c5c5 Colocated readCSV util with members input serializer
refs 5a6ce5abfe

- Allows to move files and keep the history in one go.
- 'csv-parser' will be upgraded to 'papaparse' lib in a new package
2020-06-19 18:24:51 +12:00
Nazar Gargol
5a6ce5abfe Extracted members csv input serialization logic into separate module
refs a6b5a82f09

- This is prep work for extraction into members repository.
2020-06-19 17:58:33 +12:00
Nazar Gargol
a6b5a82f09 Extracted members csv serialization logic into separate module
no issue

- This is prep work for extraction into members repository.
2020-06-19 17:23:05 +12:00
Nazar Gargol
e698f9f95c Fixed handling for grouped validation errors during members import
refs 7904c303a7

- Model's validation errors are sometimes returned as an array of errors, in those cases they should be flattened to before returning to the client. This way there is more insight into what went wrong during the import.
2020-06-19 16:34:23 +12:00
Fabien O'Carroll
4716d13d13 Added API endpoints for disconnecting Stripe
no-issue
pr: https://github.com/TryGhost/Ghost/pull/11930

- Upgraded @tryghost/members-api to 0.23.0

  This version includes a new method hasActiveStripeSubscriptions

- Added /admin/members/hasActiveStripeSubscriptions

  This can be used to determine whether or not we should allow removing
  the stripe keys.

- Added /admin/settings/stripe/connect

  This can be used to delete a Stripe Connect integration, provided
  there are not active subscriptions
2020-06-18 18:42:20 +02:00
Daniel Lockyer
ef7555f9aa Updated Ghost-Admin to v3.20.3 2020-06-18 16:20:13 +01:00
Kevin Ansfield
2553da1bee Fixed mobiledoc.populateImageSizes handling of empty image card payloads
no issue

- it's possible that image cards can be saved when they have no `src` attribute (placeholders)
2020-06-18 15:32:42 +01:00
Daniel Lockyer
3070bfa486 Updated Ghost-Admin to v3.20.2 2020-06-18 15:10:11 +01:00
Rish
87c31444fd Revert "Added new settings for members modal settings (#11927)"
This reverts commit ea7d450a9b.
2020-06-18 19:06:13 +05:30
Kevin Ansfield
4fb39f29cc Fixed population of image sizes when forcing re-render of posts
no issue

- fixed incorrect method name when calling
- fixed problem with setting post.mobiledoc to a promise rather than waiting for the size population to finish and setting it to a mobiledoc string
2020-06-18 14:02:53 +01:00
Kevin Ansfield
95525eeadc Allowed ?formats param in Admin API post+page create/edit requests
no issue

- `?formats` is useful when you want to get `html` back from the Admin API when creating/editing posts, otherwise you're forced to make a following `GET` request
2020-06-18 13:59:01 +01:00
Rishabh Garg
ea7d450a9b
Added new settings for members modal settings (#11927)
no issue

-  Adds new settings for members modal customization to default settings
- `membersjs_show_beacon` controls the visibility of beacon in members modal
- `membersjs_show_signup_name` controls the visibility of name field in signup
- `membersjs_allowed_plans` controls the visibility of plans allowed for member to signup with
-  Adds stripe connect check to determine if stripe is setup or not
-  Adds the 3 new settings to members site data
2020-06-18 17:11:08 +05:30
Daniel Lockyer
4a2114e405 Updated Ghost-Admin to v3.20.1 2020-06-17 17:13:05 +01:00
Kevin Ansfield
f27282bc3f
Added mobiledoc transform to populate missing image sizes on forced re-render (#11924)
no issue

- adds `populateImageSizes()` to our mobiledoc lib module
  - uses `image-size` lib to speed up reading of image dimensions
  - for local images, use storage adapter with same guards as used by `handle-image-sizes` middleware so that we don't insert srcsets for images that aren't transformable
  - for unsplash images, remove any width and crop params from the url so it points to the full-size image
- use `populateImageSizes(mobiledoc)` to modify post model's mobiledoc when re-rendering
2020-06-17 13:12:32 +01:00
Kevin Ansfield
4537ccd329 🐛 Fixed embed/bookmark retrieval for sites that require cookies during redirects
no issue

- known failing case was Medium. Some *.medium.com articles triggered redirects but without cookies being stored/sent across the redirect requests it would cause an infinite redirect loop and we'd abort after 10 redirects
- use `got`'s [cookie support](https://github.com/sindresorhus/got/tree/v9.6.0#cookies) via `tough-cookie` so that we can create bookmark cards for medium.com and other sites with similar problems
2020-06-17 10:03:36 +01:00
Kevin Ansfield
86f14c28a1 Fixed no-shadow linter warnings in image-size.js 2020-06-16 12:36:55 +01:00
Daniel Lockyer
dc1997fc89 Updated Ghost-Admin to v3.20.0 2020-06-16 11:55:57 +01:00
Hannah Wolfe
a9759736d6
🐛 Fixed private blogging exposing 404 and robots (#11922)
- There were various cases where it was possible to trigger a private site to display a 404 instead of redirecting to /private/
- Private mode was also not always displaying the correct robots.txt
- This PR includes tests for all cases in test/frontend-acceptance/default_routes_spec.js & where possible the unit tests have also been updated for completeness
- Fixing the 404 issues required
    - Better handling of paths using req.path instead of req.url in filterPrivateRoutes
    - Additional error handling, to cover the case that a tag/author RSS feed does not exist
 - Fixing the robots.txt required the order of middleware to be changed, so that private blogging gets a chance to render first
    - NOTE private blogging is the only app with a setupMiddleware function so nothing else is affected
2020-06-16 11:42:32 +01:00
Fabien O'Carroll
c5b8dab523 🐛 Fixed custom redirects for subdirectory setups
no-issue

This issue only occurs when using custom redirects with a subdirectory
setup, and the path to be redirected from is expressed as a regex, and
the url that is being redirected to is not an external url.

The issue has a few components:

- Redirect paths as a regex generally use the ^ to ensure that they
  match the beginning of the path.

- The path that the regex is matched against conditionally excludes the
  subdirectory, specifically, the subdirectory is excluded for external
  urls

These combined means you end up with a regex like /^\/custom-redirect/
and a path like /subdir/custom-redirect, these will not match/replace
correctly, and you'll end in an infinite redirect loop.

The fix here is to *always* remove the subdirectory when testing regex's
and then conditionally adding it back *only* for the redirect, and only
if it is an internal redirect
2020-06-16 10:24:13 +02:00
Nazar Gargol
5f00619d1a Renamed members import/export endpoints
refs cb26fd9305

- Reasoning behind the rename in referenced commit
2020-06-16 18:08:50 +12:00
Hannah Wolfe
f4641aabe4 Rename private blogging mw functions
- This is just a nicety, trying to make it easier to follow the logic of private blogging
2020-06-15 20:55:59 +01:00
Hannah Wolfe
4ae907781a Private blogging misc cleanup
This is a bunch of small changes, that simplifies working with the private blogging module:
- remove reference to really old paginated RSS behaviour
- remove handling for /rss and allow our standard redirects to redirect to /rss/ and then execute
- readd should to tests so that they can be run standalone
- fix eslint warning
2020-06-15 20:16:13 +01:00
Kevin Ansfield
e0d1763424 🐛 Fixed fallback to bookmark card when pasting localhost Ghost url
no issue

- localhost check was causing an immediate "no provider" error when fetching oembed for a localhost url and bypassing the bookmark fallback even when the configured site lives on localhost
- allow `localhost:port` through in oembed endpoints when it matches the configured url
2020-06-15 18:38:43 +01:00
Kevin Ansfield
ce8bc9d4bf Added imageOptimization:srcsets config to allow disabling of srcset attrs in content
no issue

- pass `srcsets` value through to mobiledoc renderer
  - it will stop adding `srcset` attributes to images when explicitly set to `false`
- adds `reload()` method to `mobiledocLib` so that memoized instances of the renderer can be reset during tests that change config values
2020-06-15 16:45:36 +01:00
Kevin Ansfield
73429589bd Reverted to previous user-agent for bookmark and oembed requests
no issue

- need to document changes so that external systems can adapt to the revised user-agent
2020-06-15 16:02:04 +01:00
Hannah Wolfe
27066ce910
🐛 Fixed missing text version in bulk email (#11919)
closes #11917

- Pass text-only version to mailgun as `text` not `plaintext`
- This ensures we send a text-only version of the email, and this in turn should help to improve spam scores
2020-06-15 15:31:09 +01:00
Rishabh Garg
84d0a46c6e
Removed "from address" overwrite in importer (#11914)
refs https://github.com/TryGhost/Ghost/issues/11414

- Importing data currently overwrites the existing "from address" with new value
- "from address" needs to go through email validation flow before update which was bypassed
- Updates importer to not allow overwrite for "from address" and use existing
- Adds test for "from address" overwrite
2020-06-15 12:22:55 +05:30
Hannah Wolfe
edff7ac853
Adding INR currency support (#11911)
- We have many customers asking for INR as there are special rules in Stripe for this currency
- As well as a desire for local-selling
- Meaning it's not valid to use e.g. USD instead
2020-06-12 19:36:25 +01:00
Kevin Ansfield
424e43975c Allowed ?force_reload=true through on pages edit endpoint
refs 008f86fc29

- same as posts. Related functionality to handle the query param exists on the Post model
2020-06-12 19:15:19 +01:00
Fabien O'Carroll
63f1e57036 Fixed Members Stripe config for Stripe Connect
no-issue

This was initially missed as local settings always had the original
stripeDirect keys, this ensures that regardless of Connect vs Direct vs
Both vs Neither that the config is correct.

Also ensures that the Members API instance is reloaded when the Stripe
Connect settings are changed.
2020-06-12 19:22:36 +02:00
Kevin Ansfield
008f86fc29 Added ?force_rerender=true param to posts edit endpoint
no issue

- there are various situations where we adapt/fix/improve our mobiledoc->html output over time but we didn't have a way of updating old content without manually editing the mobiledoc and saving, or running an expensive migration to re-render all old content
- this adds a `?force_rerender=true` query param to the `PUT .../admin/posts/:id/` endpoint that allows the `html` field to be re-generated without modifying the `mobiledoc` field contents
2020-06-12 18:05:57 +01:00
Kevin Ansfield
5d0d0b397e Fixed no-shadow linting warnings in models/post.js 2020-06-12 17:55:40 +01:00
Nazar Gargol
cb26fd9305 Changed members upload validation endpoint to /members/upload/validate
no issue

- The intention is to move away from using file type names in URLs. This endpoint is meant to correspond to current `POST /members/csv` endpoint, that is  planned to be renamed into `/members/upload`. And the `GET /members/csv` to be renamed to  `/members/download` respectively.
2020-06-12 23:31:56 +12:00
Rishabh Garg
3fe3814cf0
Added new meta tag for site url behind dev flag (#11906)
refs TryGhost/members.js#39

Currently, there is no way to determine exact Ghost site url from a theme, which is used by new members.js to initialize the script and use members/admin API. This change

- adds a new meta tag - `ghost:site` with value as ghost site url, when members is enabled
- new meta tag is behind dev flag along with members.js script
2020-06-12 16:58:27 +05:30
Kevin Ansfield
f4d9a41d3b
Added ?paid query parameter to Admin API members browse endpoint (#11892)
no issue

- NQL does not support the relationship setup that members->stripe customer<->stripe subscriptions uses so it wasn't possible to use the `filter` param to query against having an active subscription
- adds `customQuery` bookshelf plugin that allows customisation of SQL query used in `findPage` method by individual models
- use `customQuery` in Member model to set up joins and conditionals to select free/paid members when `options.paid` is present
- allow `?paid` param through API and permitted options for member model
2020-06-12 12:12:10 +01:00
Daniel Lockyer
2229fd4d6c Fixed import paths for errors and logging modules
no issue

- errors is now `@tryghost/errors`
- logging moved location to `core/shared/logging`
2020-06-12 09:09:46 +01:00
Nazar Gargol
7904c303a7 Added invalid import record errors and counts
no issue

- This new format allows to return additional metadata with failed import records. The data for invalid records is returned in following format:
```
{
    count: {count_of_invalid_records},
    errors: [{
      message:	"Members not imported. Members with duplicate Stripe customer ids are not allowed." // message field of the error
     context:	"Attempting to import members with duplicate Stripe customer ids." // context field of the error
     help:	"Remove duplicate Stripe customer ids from the import file, and re-run the import." // help field of the error
     count:	2 // count of this specific error
    }]
};
- Errors are grouped by their context fields because message fields sometimes can contain unique information like Stripe customer id, which would produce too many errors in case of bigger datasets.
2020-06-12 19:59:36 +12:00
Nazar Gargol
589d826afd Added /members/validate ednpoint to Admin API
no issue

- This endpoint is meant to be used for validation of imported members
- Main function at the moment is to validate if stripe_customer_id present in the dataset exists in connected Stripe account
2020-06-12 16:34:12 +12:00
Daniel Lockyer
45188907c5 Updated Ghost-Admin to v3.19.2 2020-06-11 19:55:43 +01:00
Fabien O'Carroll
4525fa65b0 Added manual check for stripe connect integration
no-issue

We had a circular dependency issue when using the members service, so
this commit replaces the use of that with a check to the settings cache
to check if stripe connect has been setup.
2020-06-11 20:21:24 +02:00
Fabien O'Carroll
2418594651 Revert "Refactored ghost_head to use isStripeConnected"
This reverts commit 0d11c66c26.
2020-06-11 20:21:24 +02:00
Daniel Lockyer
6b143e86fd Updated Ghost-Admin to v3.19.1 2020-06-11 15:43:03 +01:00
Rish
41b3ecc744 Removed extra redirect on Admin for from address update
no issue

- Ghost-Admin redirects all paths to `/ghost/settings...` as `/ghost/#/settings/...`, this updates the admin redirect on successful magic link validation to directly use the latter to avoid extra redirect
2020-06-11 19:49:19 +05:30
Rish
9c478d778a Fixed email copy for update email template
no issue

- Fixes email copy changes for update-email mails sent for from address update
2020-06-11 18:19:53 +05:30
Kevin Ansfield
70b205618f Added responsive-image srcset attribute to image and gallery card output
no issue

- adds a set of hardcoded "content image sizes" to the base config
- adjusts `handle-image-sizes` middleware to always allow the hardcoded content image sizes to be genreated
- updates `@tryghost/kg-card-factory` to allow passthrough of options to card renderers
- updates `@tryghost/kg-default-cards` to add `srcset` output for image and gallery cards
2020-06-11 13:28:44 +01:00
Fabien 'egg' O'Carroll
bfc03eb120
Replaced nql-map-key-values with @nexes/nql (#11896)
closes #11008

- Updated @nexes/nql to 0.4.0

  This version exports the mapKeyValues utility function

- Replaced nql-map-key-values with @nexes/nql util fn

  Usage was found using `rg nql-map-key-values` and replaced globally.

- Deleted nql-map-key-values module in shared 

  Now that this module isn't referenced anywhere else, we can remove it,
  relying solely on the util exported by @nexes/nql
2020-06-10 19:17:25 +02:00
Rish
82cf095600 Added stripe config check to members site data
refs https://github.com/TryGhost/members.js/issues/43

- Adds new `isStripeConfigured` flag to public members site data which denotes if stripe setup is completed
- Helps clients like members.js/themes to configure payment behavior based on this flag
2020-06-10 21:15:13 +05:30
Fabien O'Carroll
0d11c66c26 Refactored ghost_head to use isStripeConnected
no-issue

This is to make sure that whether stripeDirect is enabled or not, that
the stripe script is correctly added to ghost_head.
2020-06-10 17:08:35 +02:00
Fabien O'Carroll
5f808dfbee Encoded the mode in the Stripe Connect OAuth state
no-issue

The service at stripe.ghost.org must know which client_secret to use,
either the test, or live one. By encoding a JSON object as the state we
are able to pass data through the flow to inform this decision at the
end.

Note, that we still keep a random value in the state to protect against
CSRF attacks.
2020-06-10 14:06:57 +02:00
Fabien O'Carroll
33d1148cff Supported test mode in members_stripe_connect API
no-issue

We've added a "mode" query param to the members_stripe_connect api auth
method, allowing the client to easily switch between live and test mode.
2020-06-10 14:06:57 +02:00
Fabien O'Carroll
2ff151a893 Added support for stripe connect test mode
no-issue

This adds the ability to pass a `mode` param of 'test' to the members stripe
connect service, which will ensure we use a testmode client_id for the
Stripe Connect OAuth flow.

This will allow users to connect their account in testmode.
2020-06-10 14:06:57 +02:00
Nazar Gargol
c8351720b1 Removed levtover code
refs 81fc5f8eda

- Comment was left here unintentionally
2020-06-10 14:40:33 +12:00
Fabien O'Carroll
9348134afe Added display_name & account_id to stripe connect
no-issue

This will allow us to display this information in the Admin to make it
easier for the user to see what the current state is.
2020-06-09 17:11:17 +02:00
Kevin Ansfield
8c235ff766 Shortened insufficient oembed data error message
no issue

- improves display in client
2020-06-09 12:47:34 +01:00
Nazar Gargol
81fc5f8eda Added special handling for member import with Stripe connection
no issue

- When imported member contains stripe_customer_id data but there is no Stripe configured on the Ghost instance such import should faiil. The logic is consistent with one where import fails after not being able to find customer in linked Stripe account
- Fixed import stats to show import failures instead of "duplicate" when the validation error is of "Stripe" origin
2020-06-09 23:02:38 +12:00
Daniel Lockyer
d527f0be51 Updated Ghost-Admin to v3.19.0 2020-06-09 10:27:45 +01:00
Rish
c63f7f2b13 Updated copy for email address change template
no issue

- Fixes typo and title for email address update mails
2020-06-09 12:11:57 +05:30
Nazar Gargol
dba664a200 Removed unused variable
refs 53abf79869
2020-06-09 15:31:42 +12:00
Nazar Gargol
53abf79869 Removed format-csv module in favor of papaparse
no issue

- The aim was to extract format-csv to become an external dependency. After some analysis found out that native papaparse method `unparse` was achieving the same results with a lot better test coverage and stability. Because papaparse will become Ghost's default csv processor in near future decided to integrate papaparse instead of extracting module for the format-csv module's code, which would become redundant soon anyways.
- For reference papaparse will substitute current csv-parser lib because it's better performance and maturity.
- Performance comparison can be checked here - https://github.com/Keyang/csvbench#result . At the time of writing papaparse is rougly 40% faster than csv-parser
2020-06-09 15:23:15 +12:00
Daniel Lockyer
5d30af1513 Fixed wrong require in post email serializer
refs https://github.com/TryGhost/Ghost/pull/11807

- this was originally reported in the PR above, but we could not
  reproduce it on master
- presumably the user had the latest version of moment installed for
  other purposes and so they were seeing the issue
- between moment 2.24.0 and 2.26.0, something must have changed which
  stopped the previous functionality working
2020-06-08 19:43:55 +01:00
Rish
bca41e1877 Allowed updating from address domain for member emails
refs https://github.com/TryGhost/Ghost/issues/11414

Confirms if the fromAddress for sending member emails is valid and accessible using magic link flow, allowing owners to update full from address including domain change.

- Extends member service to handle magic link generation and validation for email update
- Updates existing setting endpoint to not directly update from address
- Adds new endpoint to send magic link to new address
- Adds new endpoint for validating the magic link when clicked and update the new email for from address
- Adds new email template for from address update email
2020-06-09 00:06:07 +05:30
Rish
4200eaf1f7 Added migration updating members from address with domain
refs https://github.com/TryGhost/Ghost/issues/11414

We want to allow adding custom domains to member's from address as long as the new email is verified using the magic link flow and not saved directly to DB. Since we currently don't store domain as part of fromAddress, this PR -

- adds migration to update all existing fromAddress by appending site domain
2020-06-08 23:47:02 +05:30
Rish
bbcd32d204 🐛 Fixed "from" parameter handling for bulk email
closes https://github.com/TryGhost/Ghost/issues/11768

- Wraps from parameter in double quotes so mail clients can read it whole
- Escapes double quotes in site title to avoid clash with wrapping
2020-06-08 21:56:11 +05:30
Hannah Wolfe
d541a14826 Change theme uploads to move & delete at end
- Currently theme uploads delete the existing theme before copying the new files into place
- If something goes wrong with the delete action, you will end up in a bad state
   - Some or all of the files may be deleted, but now Ghost won't try to put the new theme in place, instead returning an error
   - This leaves you with an invalid active theme and a broken site
- Unlike delete, move is a one-hit operation that succeeds or fails, there moving a theme is safer than deleting
- This updated code moves the old theme to a folder with the name [theme-name]-[uuid] before copying the new theme into place
- Even if this fails, the files should not be gone
- There's a cleanup operation to remove the theme backup at the end, but we don't care too much if this fails
2020-06-08 16:12:17 +01:00
Kevin Ansfield
db68560b11 🐛 Improved error message output when oembed request fails
refs https://github.com/TryGhost/Ghost/issues/11212

- if a bookmark card fetch is performed (either directly or from fallback) and the page does not have an extractable title, return a more specific error message than "No provider found for supplied URL."
2020-06-08 15:06:00 +01:00
Kevin Ansfield
5aa6a3dbad Fixed no-shadow linting errors in oembed controllers 2020-06-08 12:52:41 +01:00
Kevin Ansfield
d8773dcc45 Allowed bookmark cards to be created without a description
no issue

- title+description was too limiting for some sites so we're relaxing the requirements to only require a title
2020-06-08 07:55:38 +01:00
Nazar Gargol
c7648737ca Added context and help message to member linking with Stripe account error
no issue

- When the customer cannot be imported because they are missing from linked Stripe account or the linked account is incorrect one, these new messages should provide a better clue about what has caused the error and how to act on it.
2020-06-06 00:06:19 +12:00
Nazar Gargol
633ba27f0e Added custom label assignment to imported members
no issue

- There is a need to be able to label certain import group of members
with custom labels. This will allow to distinguish/filter these newly
imported  members.
- Allowed `POST /members/csv/` endpoint to accept `labels`
field parameter which assigns labels to every member from imported csv.
2020-06-05 16:22:09 +12:00
Kevin Ansfield
fdeb7daf40 Added shared_views default setting
no issue

- we want custom views to be shared for all users and managed by admins rather than per-user
2020-06-04 21:29:06 +01:00
Nazar Gargol
12c8b63a4a Added more specific error handling when adding duplicate labels
no issue

- Similarly to other additive api methods  (e.g. members.add) returned more specific ValidationError with contex filled in with the reason why adding did not succed.
- This change is needed for more graceful label handling when adding new members through import
2020-06-05 00:23:10 +12:00
Nazar Gargol
59c773fb04 Removed cache invalidation header when adding label through Amdin API
no issue

- Adding labels doesn't cause any content to invalidate, similarly to adding members. Unlike it's caunterpart - tags, there is no dependent "frontend" content that would become invalid
2020-06-04 23:44:19 +12:00
Daniel Lockyer
eeea5ef64d Updated Ghost-Admin to v3.18.1 2020-06-02 18:07:38 +01:00
Fabien O'Carroll
7c188ec1eb 🐛 Fixed settings controller edit method
no-issue

This was because the API was recieving a default `null` value for the
stripe_connect_integration_token setting.

This also improves the logging for this error.
2020-06-02 19:01:19 +02:00
Daniel Lockyer
a7ce7fcdc1 Updated Ghost-Admin to v3.18.0 2020-06-02 15:22:02 +01:00
Fabien O'Carroll
37a1947fc5 Fixed broken regression tests from #11861
no-issue

The config is now a class and the context of `this` was not bound
2020-06-02 16:20:57 +02:00
Kevin Ansfield
4e139ee5b4 Fixed linting 2020-06-02 14:33:43 +01:00
Kevin Ansfield
64ed246d03
Merge pull request from GHSA-4m2q-w26j-h268
no issue

- added an `externalRequest` lib
  - uses same underlying `got` module as our `request` lib
  - uses `got`'s `beforeRequest` and `beforeRedirect` hooks to perform it's own dns resolution for each url that's encountered and aborts with an error if it resolves to a private IP address block
  - includes a bypass for Ghost's configured url so that requests to it's own hostname+port are not blocked
- updated v2 and canary oembed controllers to use the `externalRequest` lib
2020-06-02 14:30:10 +01:00
Fabien O'Carroll
44f1c0a7b6 Updated members config to respect stripeDirect
no-issue

If the stripeDirect config value is NOT set / false, then connect token
should be used over the API keys if both are present, or else use
whichever is present.  If the stripeDirect config value is set /
true,then API keys should be used and stripe connect token should be
completely ignored.

The reason for this is that we want to use Stripe Connect as the primary
auth method going forward, but we want to keep the direct version
availiable should the Ghost Foundation ever dismantle. This will allow
people to use all the same features with no dependency on an external
service.
2020-06-02 15:28:42 +02:00
Fabien O'Carroll
a1f883edbc Refactored members config to use DI
no-issue

This makes testing it much easier
2020-06-02 15:28:42 +02:00
Fabien O'Carroll
fc0e97593b Added membersStripeConnect controller auth method
no-issue

In order to issue a redirect we need access to the "raw" req/res
objects, which is why we must return the function which gets access to
them.

The members service is used to create the auth url and to update the
users session.
2020-06-02 15:28:42 +02:00
Fabien O'Carroll
5b058d6d5b Added permission for auth:members_stripe_connect
no-issue

This permission is used to ensure that only the Owner can
authenticate/authorize the members_stripe_connect integration.
2020-06-02 15:28:42 +02:00