no issue
- When processing entries with new labels in parallel Bookshelf relations is trying to create them which caused unique key constraints to fail. To avoid the failure, all labels should be pre-created before proceeding with creating members
- Member limit code was duplicated in 2 places unnecessarily
- Also used member api code that fetched members and subscriptions fully hyrated when we only need a count
- Using a raw query significantly improves performance here
no-issue
This updates the Admin API Member resource to *not* cancel subscriptions
by default, and adds a `cancel` option. This can be used over HTTP by
including a `cancel=true` query parameter.
- mailgun has a testmode flag we can use to get email to be accepted but not delivered
- this is useful for developers testing general bulk email code - not for users - so it is only available via config
no-issue
Up until now we have left orphaned rows in members_stripe_* tables when
a member is deleted, this updates the destroy method so that we cascade
and remove any MemberStripeCustomer and StripeCustomerSubscription
models related to the Member.
This also adds regression tests for the new functionality as well as to
confirm the existing functionality of cascading to the members_labels
join table
This adds the relations of Subscription->Customer & Customer->Member
refs https://github.com/TryGhost/members.js/issues/72
- Portal is using using publication logo from settings for signup/signin pages
- Instead, we are switching to using publication icon from settings, which also needs to be passed in site data API
no issue
Having all members created during an import labelled with a specific "import label" is useful for later operations such as bulk delete/edit or simply recording how and when a member was created.
- automatically create a label with the date/time the members CSV import occurred and assign it to all imported members
- return the import label data in the API response so that clients can react accordingly such as automatically filtering the members list by the label once an import finishes
refs #12074
Since we've split members settings into multiple keys the
reconfiguration of the members-api has been happening in quick
succession as the stripe_connect_* settings are all set at once.
This debounces the call to reconfigure the members-api so that we only
need to instantiate it once.
no issue
- By default, GhostMailer throws EmailError with statusCode as `500` for any failure in sending mail
- In case of failure due to `RecipientError`, status code as now correctly sent as `400` as its a bad request and not an error we can't handle.
closes#12049
Stripe plans used to default to 0, and our new validation of plan
amounts were causing issues when importing from an older version of
Ghost, this updates the validation to be skipped when importing.
- Added regression test for importing plans
refs https://github.com/TryGhost/Ghost/issues/11971
- Added statusCode from bulk email provider to API response
- Updated error messages for different bulk email(mailgun) failure states
- Added `context` to preview mail API error message with mail provider's error message
closes#12033
- Added webhooks schemas and definitions.
- Added validation checking if integration_id is present when using session auth. This is needed to prevent orphan webhooks.
- Integrated webhook schemas into frame's validation layer.
- Added isLowerCase ajv keyword support. This is needed to be able to do isLowerCase validation using JSON Schema for webhooks.
closes#12001
* Moved settings validation to the model
This moves the settings validation out of the validation file and into
the model, as it is _only_ used there.
It also sets us up in the future for custom validators on individual
settings.
* Improved validation of stripe_plans setting
- Checks `interval` is a valid string
- Checks `name` & `currency` are strings
* Moved stripe key validation into model
The stripe key settings are all nullable and the regex validation fails
when the input is `null`. Rather than reworking the entirety of how we
validate with default-settings validation objects, this moves the
validation into methods on the Settings model.
* Added tests for new setting validations
Adds tests for both valid and invalid settings, as well as helpers
making future tests easier and less repetitive
refs #11980
This adds a new handlebars helper {{amp_analytics}} which outputs the
GA configuration script if the amp_gtag_id setting is present.
We also update {{amp_components}} to output the GA script in the head if
the setting is present
closes#12015
refs 95880dddeb
- The bug was caused by falsy plaintext field assignment to empty string `''` when the html content was `null`. Because of the `setEmptyValuesToNull` function (referenced commit), there is no sense to assign empty string value to plaintext property, because it would still end up being `null`
- The `''` -> `null` conversion was confusing the model layer to think that some fields were changed, where in reality none did. This in turn lead to a bug with falsy cache invalidation
closes#12016
- The change detection didn't work when editing post_meta fileds because we only check current model's `_changed` fields when performing `wasChanged()` check
- A solution was adding change tracking of post_meta relation to currently edited post model and overloading `wasChanged` method to check these fields as well
refs #12043
- On updating From-address from Members settings in Labs, we send a confirmation email to the updated address with magic link for verification
- Previously, no explicit sender email was being set for this so fallback config address was used
- This updates the sender address to use the current from address for "from-address" update emails
no-issue
- Added breaking test for webhook url including subdirectory
- Previously the webhook handler URL was generated incorrectly when
running Ghost on a subdirectory, appending the path to the root of the
host, this fix ensures that the subdirectory is included before the
path.
no-issue
This version of members-api includes changes to how webhooks are
managed, previously they would be deleted and recreated on every boot of
Ghost. Now they are created and the secret is persisted, on boot the
webhook is updated to the most current url and events. If the api
version is wrong or the update fails, the webhook is deleted and
recreated and the settings updated.
- Installed @tryghost/members-api@0.24.0
- Updated config to work with 0.24.0
no-issue
They will be used to store webhook information so that we can persist it between
boots and simplify the creation process of webhooks in members
closes https://github.com/TryGhost/Ghost/issues/12026
- 3 new portal settings were added in `3.23` in default settings - `portal_button_style`, `portal_button_icon`, `portal_button_signup_text`
- New settings default to group `core` on migrating from pre 3.22 versions due to missing columns in DB
- Migration here updates the new settings to correct group
no-issue
- `canonicalUrl`
- Updated to use `canonical_url` & fall back to previous functionality
- `ogTitle`
- Updated to use `og_title` and fall back to previous functionality
- `ogImage`
- Updated to use `og_image` and fall back to previous functionality
- `ogDescription`
- Updated to use `og_description` and fall back to previous functionality
- `twitterTitle`
- Updated to use `twitter_title` and fall back to previous functionality
- `twitterImage`
- Upated to use `twitter_image` and fall back to previous functionality
- `twitterDescription`
- Updated to use `twitter_description` and fall back to previous functionality
closes#11994
- Adds support for ordering based on slug filter that contains a slug-is-in filter. It is applied only to Content API's resources - post, page, tag, author. The order is applied in the same order in which slugs appear in the filter.
- For, example providing following query parameter filter for any of the above resources: `?filter=slug:[kitchen-sink,bacon,chorizo]`, would filter them by these slugs and order in the same way defined in the filter
- Can be used in handlebars templates in following way: `{{#get "tags" filter="slug:[slugs,of,the,tags,in,order]"}}`
- The property conteining this new order is assigned to `autoOrder` instead of `rawOrder` intentionally. This explicit asstignment would allow distinguishing where the 'orderRaw' comes from the model or the API layer. Apart from adding necessary context this separation makes it easier to refactor separately model layer and API specific ordering in the future
- This commit also fixes default filtering for `author` resource in Content API. The serializer was never used before as it was missing from `serializers/index.js` module.
no issue
- Currently both old and new membersjs script are getting injected behind dev flag
- New script handles the current script behavior as well making its load redundant when on dev flag
- Removes old script load in favor of new
no issue
- Changes introduced to both API v3 and v2
- Makes sure to use the same integration_id as authenticated integration for the webhook's data.
- Makde it is impossible to create orphaned webhooks using token authentication
- Allowed only parent integration to edit it's children webhooks. Throwing permission error otherwise
no-issue
There is concern that the settings cache can return `null` for values
which it cannot parse correctly, this just ensures that we always have
an array where we expect one
no issue
- Existing and new sites should have portal button hidden by default and need explicit switch on from Admin.
- Default value for new sites has been updated, but existing sites still have old default and portal button set as true
- Migration here resets portal button setting to false for all existing sites, and they'll need to be explicitly turn it on from admin again
no issue
- Adds new portal button settings to members site data for portal script
- Updates settings input/output serializers to handle portal icon image url
no issue
- Adds new portal settings - `portal_button_style`, `portal_button_icon` and `portal_button_signup_text`
- New settings allows customization of portal button
- Updates tests to include new settings
* tag '3.22.2':
v3.22.2
Updated Ghost-Admin to v3.22.2
Emitted all settings events on reinit of cache (#12012)
🐛 Updated access to be true by default in v3 API
Hardened members subscription migration against missing data (#12009)
closes#12003
There are a few parts of Ghost that rely on the settings events being
emitted anytime a setting is changed, so that the data is kept in sync.
When a setting is renamed in a migration essentially what happens is
that the settings value is changed from a default value to its actual
value, but this does no emit an event.
Anything that is initialised before migrations have run that relies on
the events to keep it up to date will have stale data - e.g. the themes
i18n service.
This change ensures that when we `reinit` after migrations have been
run, we emit events for every setting to tell the rest of Ghost that it
has changed.
closes#11990
- access should be a members feature, but it was already accidentally exposed to the theme layer
- it has now been added to the API even if members is disabled
- access defaults to true, unless members is enabled
- when members is enabled, access is set to the currently logged in members' access
closes#11993
We had some issues with some databases being in an unexpected state this
check for each property before using it add uses defaults when it is
missing.
no issue
- when searching for paid/free members, the `members_stripe_customers`
table would be joined into the query on `members`
- this table also has a `name` and `email` field, so both MySQL and
SQLite would complain about ambiguous fields in the query
- the result of this would be a 500 error thrown inside Ghost, and no
useful response to the user
- this commit explicitly chooses the `members` table to check against,
and also adds a test for this
no issue
- The membersjs default trigger button was set to be visible by default, even behind dev flag
- This changes the default visibility to false so it needs to be switched on from Admin to be visible on screen
closes https://github.com/TryGhost/Ghost/issues/11944
- updates `@tryghost/image-transform` to version that exposes `canTransformFiles()` which checks for `sharp` availibility
- updates `@tryghost/kg-default-cards` to version that accepts a `canTransformImage()` method as an option
- updates our `mobiledoc` lib to pass a `canTransformImage()` function that returns false if sharp is unavailable, the image extension is not supported, or the storage engine in use does not support image transforms
- updates `populateImageSizes` to fetch image sizes when transforms are unavailable as the render/not-render is now handled in the renderer and we don't need to worry about adding size information to the mobiledoc source
refs https://github.com/TryGhost/Ghost/issues/10318
- re-initialize settings cache after migrations by shutting down to clean up event listeners then and calling `init` again
- important to ensure `db.ready` event is not emitted until settings have finished re-initializing to avoid problems with background processes using the db connection which is disconnected/re-connected or being kicked off with out-of-date settings
refs 05f6faf846
- The logic in "Stripe" error detection depended on error's message string matching, which is not a good practice in general. Had fixed it do to exact match on "context" of the error we throw internally and left more extensive comment about why things are implemented the way they are.
no issue
Output serializer's url util was expecting `og_image` and `twitter_image` to be top-level attributes in the `attrs` object but they are actually nested under `posts_meta`.
- updated the code to use lodash's `get/set()` so that we can work with paths for easier handling of nested objects
- fixed unit tests where the mocked data under test did not match real-world data
closes#11932
- as per the issue, there is no need to output a <link> tag if the favicon is the default /favicon.ico, as all browsers automatically check for this
- instead the favicon <link> is only output if a custom favicon has been set, telling the browser to look somewhere different to its default location
- some of the tests expected 3 links in the html head, but through the favicon change there are often only 2.
Co-authored-by: RenCloud <rencloud@pop-os.localdomain>
closes#11574
- the current implementation of the access property has it frontend only, and wired up only in one place
- this leaves it only available in a handful of places, e.g. can't use it in a post loop or get helper
- the current implementation also fails logically if the html content of the post is blank
This fix moves the behaviour to the API
- this ensures the field is always available no matter what context you are in
- it also updates the logic to use the same membersHasAccess logic as is used to gate the post, so it's always correct
TODO: should reconsider the location of this code
refs https://github.com/TryGhost/Ghost/issues/10318
- Skipped `members_subscription_settings` and `stripe_connect_integration` from settings import
- Fixed import data field filtering for core and theme groups
refs https://github.com/TryGhost/Ghost/issues/10318
- Updates `boolean` serialization in v2/canary serializers to apply only for `boolean` type settings
- Updates `boolean` transformation in model layer `format`/`parse` to check on `boolean` type setting
- Removes error thrown on Read-only setting for settings edit endpoint
- Updates v2/canary input serializers to remove any Read-only settings (using RO flag) to avoid edits
- Added type/group mappings in the importer when pre-migration settings table import data is present
- Updates tests
refs https://github.com/TryGhost/Ghost/issues/10318
- Updates default settings to contain correct type and validation for each setting
- Updates `populateDefaults` to correctly insert type value for new settings
- Updates settings schema to allow only select types - `array`, `number`, `boolean`, `string`
- `object` is a temporary type allowed till we get rid of all JSON object settings
refs #10318
Because settings are not populated with the correct group and flags, we
must _always_ set these. Then we can check to see if there are values
which need migrating, and if not, can safely exit and leave the values
as default.
refs #10318
If migrating from a previous version that does not include the setting
being migrated from we can safely not update the new setting, and just
rely on its default value being present. When rolling back we can use
defaults if the new setting does not exist.
closes#11936
- the driving reason for doing this is so that when posts and taxonomies all have the same route,
e.g. /:slug/ the /edit route won't default to looking up tags
this is the issue outlined in #11936
- the side effect of this is that in this case, all routes will default to looking up posts before tags,
so if you have a /test/ post and a /test/ tag, prior to this change, the tag would have been loaded first
but posts are more important than tags, and should have higher priority, so this change is considered correct
refs https://github.com/TryGhost/Ghost/issues/10318
- Updates the type field for a setting to specify its type explicitly
- Type can be one of (string, number, boolean, array)
- `object` is allowed a temp type till all the object settings are removed
- existing `type` value for setting was misleading and referring to `group`
- we moved the current `type` value for a setting to `group` here - 4a9e57c170
v3.21.1
* tag '3.21.1':
v3.21.1
Updated Ghost-Admin to v3.21.1
Un-skipped populateImageSizes subdir test
Fixed unit tests interdependency issue
Fixed `populateImageSizes` handling of images in subdir setups
🐛 Fixed potentially squashed images in image cards
Refactored site url data attribute for new members script
* Updated members default settings
ref #10318
This pulls out the members_subscription_settings & stripe_connect_intgration settings into separate keys
* Updated usage of members_from_address
* Updated stripe_connect usage
* Updated members config to use new settings
* Updated members middleware to use isStripeConnected
* Updated members service to reload correctly
We reload the members-api instance when the related settings change, so
this makes sure we're listening to the correct settings changes
* Updated ghost_head helper to use new settings
* Updated theme middleware to use new settings
* Renamed members_allow_signup -> members_allow_free_signup
* Fixed tests after settings refactor
* Removed from direct key settings key
* Fixed regression tests for settings api
refs #10318
As populateDefaults is run _before_ migrations, the new settings will
already be inserted in the database, so we just need to update their
values and then delete the old settings.
refs https://github.com/TryGhost/Ghost/issues/10318
`Settings.populateDefaults()` is run before migrations during Ghost's startup. This can cause problems when new settings table columns are added (and populated in `default-settings.json`) because `populateDefaults()` was using the model layer which assumes that those columns are available, resulting in `ER_BAD_FIELD_ERROR: Unknown column` type errors.
- query the database for the available `settings` table columns
- switch to using raw knex queries without Bookshelf for insertions so that we're in control of the columns that are added
- use `_.pick` to skip any properties in `default-settings.json` that do not match to an available column - those columns will be added and populated by later migrations
- moving away from using the model to insert settings has the side-effect of not emitting `settings.added/edited` and `settings.x.added/edited` events, this should be fine because `populateDefaults()` is called before anything else is set up and listening
- added a call to `populateDefaults()` in our knex-migrator "before migration" hook so that we have consistent db state across both startup initialised migrations and manually triggered knex migrations
no issue
- Import error grouping wasn't working correctly when error didn't have defined context property
- The copy will be refined in final design review
no issue
- when using subdirectories, images can be stored in the database both with and without the subdirectory prefix. We weren't taking that into account and so images without the subdirectory were not having the `/content/images/` prefix removed when passed to the storage adapter resulting in the storage adapter not finding the image
refs https://github.com/TryGhost/members.js/issues/54
- Previously new meta tag was added which defined the site url which was used by new script
- Removes meta tag and instead directly uses a data attribute defined on script tag which it can read on load, making it much simpler
no-issue
These utils are the first steps toward getting the models out of our
migrations! The utils here interact directly with the database and where
possible don't reinvent the wheel, by using smaller building blocks to
build more comples ones.
refs TryGhost/Ghost#10318
refs 8fc526ff6
- This is symetric change to one done for v3 API (commited as 8fc526ff6)
- Added 'core' filtering for v2 API controller
refs https://github.com/TryGhost/Ghost/issues/10318
- `shared_views` is always an array (Ghost-Admin has handling to revert it to an array if it's anything else) but it's default value was set to an empty object
refs https://github.com/TryGhost/Ghost/issues/10318
- `members_public_key` and `members_private_key` are now correctly grouped under `core` which is already filtered out so we don't need the extra filter to exclude them
refs https://github.com/TryGhost/Ghost/issues/10318
refs 1dc0405803
- Adds 1:1 mapping for filtering options to renamed settings "type" to "gorup"
- Ignores the name changes and any old types
- Detailsed type -> group mappings can be checked in the refereneced migration commit
refs https://github.com/TryGhost/Ghost/issues/10318
- maps old `settings.type` values to new `settings.type/group` values
- uses an explicit map so that we don't lose information and can safely roll back even though we're modifying `settings.type` too
- updates `settings.type` values too to keep code working while we switch to using `settings.group`
- sets the `settings.group` value for all settings which are keeping the same group as their current type
- adapts `settings.type` validations to match new groups
- adds flags to specific settings, both in the migration for existing settings records and in default-settings.json for new settings records
refs https://github.com/TryGhost/Ghost/issues/10318
- `group`
- to replace the `type` column, provides a more descriptive name for the columns use
- for existing sites it will be populated by migrating data from the `type` column in a later migration
- for new sites a minimal update has been added to `parseDefaultSettings()` to populate the `group` field when settings are created during startup - fixes the NOT NULL constraint on `settings.group`
- `flags`
- signifies special handling that is different to other settings in a group
- eg, `PUBLIC,RO` would indicate that the setting is available via unauthenticated endpoints and is read-only
refs https://github.com/TryGhost/Ghost/issues/10318
- There was a copy/paste error and we didn't have a test to pick it up. Will follow up with a regression test to make sure it doesn't happen again
refs #10318
refs 2614565d5a
- Adds importer mapping for fields as in referenced migration
- The intention is to allow exports from v2/v3 to still be compatilbe with current version
- Adds a mapper for deprecated fields and imports them with new settings "keys"
refs #10318
refs 2614565d5a
- Renamed ghost_head/ghost_foot in settings to match the new names
introduced in migrations
- Above change lead to reshufling in the mappings in input/output
serializers
- Makes sure change is compatible with v2 API
refs https://github.com/TryGhost/Ghost/issues/10318
refs 2614565d5a
- Renames to match referenced migration renames
- Fixed API responses so they are consistent with newly renamed fields
- Not returning lang and timezone keys from settings in API v2 ther rest should be returned in API v3/canary
refs https://github.com/TryGhost/Ghost/issues/10318
- precursor to migrating from `settings.type` to `settings.group`
- renames `blog` type to `site`
- renames `bulk_email` type to `email`
- moves settings out of `site` (previously `blog`) into more appropriate groups such as `core` or individual feature groups
no issue
- Previous migration in commit - 2614565d5a - was using incorrect json structure for brand -> primary_color
- Fixes the up/down migrations to work on correct existing `brand: primary_color` structure
refs https://github.com/TryGhost/Ghost/issues/10318
- Adds following renames to settings table keys:
'default_locale' -> 'lang'
'active_timezone' -> 'timezone'
'ghost_head' -> 'codeinjection_head'
'ghost_foot' -> 'codeinjection_foot'
'brand.publicationColor' -> 'accent_color'
- The renames are done to match revised naming conventions and naming
exposed through APIs
- Supersedes this revert - 1eeb5a60b8 and #11946
no issue
- when using subdirectories, images can be stored in the database both with and without the subdirectory prefix. We weren't taking that into account and so images without the subdirectory were not having the `/content/images/` prefix removed when passed to the storage adapter resulting in the storage adapter not finding the image
refs https://github.com/TryGhost/Ghost/issues/10318
- Adds following renames to settings table keys:
'default_locale' -> 'lang'
'active_timezone' -> 'timezone'
'ghost_head' -> 'codeinjection_head'
'ghost_foot' -> 'codeinjection_foot'
'brand.publicationColor' -> 'accent_color'
- The renames are done to match revised naming conventions and naming
exposed through APIs
refs #10318
refs https://github.com/TryGhost/Ghost/pull/11942
- Removes force_i18n, permalinks, and members_session_secret usage from the codebase
- We deprecated these flasgs and have not used since Ghost v2. It's good time to remove them before we introduce bigger changes to how `settings` table opeartes.
- Fixed importer test. The test was meant to check if string values were converted properly, the check agains boolean didn't make much sense in this context, so removed it.
- Following this change are going to come ralated migrations to fix existing data (see ref)
refs https://github.com/TryGhost/members.js/issues/54
- Previously new meta tag was added which defined the site url which was used by new script
- Removes meta tag and instead directly uses a data attribute defined on script tag which it can read on load, making it much simpler
no issue
- reverts commit 87c31444fd but with modifications to settings naming
- Adds new settings for members modal customization to default settings
- `portal_button` controls the visibility of beacon in members modal
- `portal_name` controls the visibility of name field in signup
- `portal_plans` controls the visibility of plans allowed for member to signup with
- Adds stripe connect check to determine if stripe is setup or not
- Adds the 3 new settings to members site data
- Updates to snake case naming for members site API data
no issue
- Moves out CSV parsing and serialization related code into separate package as a part of push to modularize Ghost repo.
- Next up is to remove `csv-parser` dependency from this new package
refs 7904c303a7
- Model's validation errors are sometimes returned as an array of errors, in those cases they should be flattened to before returning to the client. This way there is more insight into what went wrong during the import.
no-issue
pr: https://github.com/TryGhost/Ghost/pull/11930
- Upgraded @tryghost/members-api to 0.23.0
This version includes a new method hasActiveStripeSubscriptions
- Added /admin/members/hasActiveStripeSubscriptions
This can be used to determine whether or not we should allow removing
the stripe keys.
- Added /admin/settings/stripe/connect
This can be used to delete a Stripe Connect integration, provided
there are not active subscriptions
no issue
- fixed incorrect method name when calling
- fixed problem with setting post.mobiledoc to a promise rather than waiting for the size population to finish and setting it to a mobiledoc string
no issue
- `?formats` is useful when you want to get `html` back from the Admin API when creating/editing posts, otherwise you're forced to make a following `GET` request
no issue
- Adds new settings for members modal customization to default settings
- `membersjs_show_beacon` controls the visibility of beacon in members modal
- `membersjs_show_signup_name` controls the visibility of name field in signup
- `membersjs_allowed_plans` controls the visibility of plans allowed for member to signup with
- Adds stripe connect check to determine if stripe is setup or not
- Adds the 3 new settings to members site data
no issue
- adds `populateImageSizes()` to our mobiledoc lib module
- uses `image-size` lib to speed up reading of image dimensions
- for local images, use storage adapter with same guards as used by `handle-image-sizes` middleware so that we don't insert srcsets for images that aren't transformable
- for unsplash images, remove any width and crop params from the url so it points to the full-size image
- use `populateImageSizes(mobiledoc)` to modify post model's mobiledoc when re-rendering
no issue
- known failing case was Medium. Some *.medium.com articles triggered redirects but without cookies being stored/sent across the redirect requests it would cause an infinite redirect loop and we'd abort after 10 redirects
- use `got`'s [cookie support](https://github.com/sindresorhus/got/tree/v9.6.0#cookies) via `tough-cookie` so that we can create bookmark cards for medium.com and other sites with similar problems
- There were various cases where it was possible to trigger a private site to display a 404 instead of redirecting to /private/
- Private mode was also not always displaying the correct robots.txt
- This PR includes tests for all cases in test/frontend-acceptance/default_routes_spec.js & where possible the unit tests have also been updated for completeness
- Fixing the 404 issues required
- Better handling of paths using req.path instead of req.url in filterPrivateRoutes
- Additional error handling, to cover the case that a tag/author RSS feed does not exist
- Fixing the robots.txt required the order of middleware to be changed, so that private blogging gets a chance to render first
- NOTE private blogging is the only app with a setupMiddleware function so nothing else is affected
no-issue
This issue only occurs when using custom redirects with a subdirectory
setup, and the path to be redirected from is expressed as a regex, and
the url that is being redirected to is not an external url.
The issue has a few components:
- Redirect paths as a regex generally use the ^ to ensure that they
match the beginning of the path.
- The path that the regex is matched against conditionally excludes the
subdirectory, specifically, the subdirectory is excluded for external
urls
These combined means you end up with a regex like /^\/custom-redirect/
and a path like /subdir/custom-redirect, these will not match/replace
correctly, and you'll end in an infinite redirect loop.
The fix here is to *always* remove the subdirectory when testing regex's
and then conditionally adding it back *only* for the redirect, and only
if it is an internal redirect
This is a bunch of small changes, that simplifies working with the private blogging module:
- remove reference to really old paginated RSS behaviour
- remove handling for /rss and allow our standard redirects to redirect to /rss/ and then execute
- readd should to tests so that they can be run standalone
- fix eslint warning
no issue
- localhost check was causing an immediate "no provider" error when fetching oembed for a localhost url and bypassing the bookmark fallback even when the configured site lives on localhost
- allow `localhost:port` through in oembed endpoints when it matches the configured url
no issue
- pass `srcsets` value through to mobiledoc renderer
- it will stop adding `srcset` attributes to images when explicitly set to `false`
- adds `reload()` method to `mobiledocLib` so that memoized instances of the renderer can be reset during tests that change config values
closes#11917
- Pass text-only version to mailgun as `text` not `plaintext`
- This ensures we send a text-only version of the email, and this in turn should help to improve spam scores
refs https://github.com/TryGhost/Ghost/issues/11414
- Importing data currently overwrites the existing "from address" with new value
- "from address" needs to go through email validation flow before update which was bypassed
- Updates importer to not allow overwrite for "from address" and use existing
- Adds test for "from address" overwrite
- We have many customers asking for INR as there are special rules in Stripe for this currency
- As well as a desire for local-selling
- Meaning it's not valid to use e.g. USD instead
no-issue
This was initially missed as local settings always had the original
stripeDirect keys, this ensures that regardless of Connect vs Direct vs
Both vs Neither that the config is correct.
Also ensures that the Members API instance is reloaded when the Stripe
Connect settings are changed.
no issue
- there are various situations where we adapt/fix/improve our mobiledoc->html output over time but we didn't have a way of updating old content without manually editing the mobiledoc and saving, or running an expensive migration to re-render all old content
- this adds a `?force_rerender=true` query param to the `PUT .../admin/posts/:id/` endpoint that allows the `html` field to be re-generated without modifying the `mobiledoc` field contents
no issue
- The intention is to move away from using file type names in URLs. This endpoint is meant to correspond to current `POST /members/csv` endpoint, that is planned to be renamed into `/members/upload`. And the `GET /members/csv` to be renamed to `/members/download` respectively.
refs TryGhost/members.js#39
Currently, there is no way to determine exact Ghost site url from a theme, which is used by new members.js to initialize the script and use members/admin API. This change
- adds a new meta tag - `ghost:site` with value as ghost site url, when members is enabled
- new meta tag is behind dev flag along with members.js script
no issue
- NQL does not support the relationship setup that members->stripe customer<->stripe subscriptions uses so it wasn't possible to use the `filter` param to query against having an active subscription
- adds `customQuery` bookshelf plugin that allows customisation of SQL query used in `findPage` method by individual models
- use `customQuery` in Member model to set up joins and conditionals to select free/paid members when `options.paid` is present
- allow `?paid` param through API and permitted options for member model
no issue
- This new format allows to return additional metadata with failed import records. The data for invalid records is returned in following format:
```
{
count: {count_of_invalid_records},
errors: [{
message: "Members not imported. Members with duplicate Stripe customer ids are not allowed." // message field of the error
context: "Attempting to import members with duplicate Stripe customer ids." // context field of the error
help: "Remove duplicate Stripe customer ids from the import file, and re-run the import." // help field of the error
count: 2 // count of this specific error
}]
};
- Errors are grouped by their context fields because message fields sometimes can contain unique information like Stripe customer id, which would produce too many errors in case of bigger datasets.
no issue
- This endpoint is meant to be used for validation of imported members
- Main function at the moment is to validate if stripe_customer_id present in the dataset exists in connected Stripe account
no-issue
We had a circular dependency issue when using the members service, so
this commit replaces the use of that with a check to the settings cache
to check if stripe connect has been setup.
no issue
- Ghost-Admin redirects all paths to `/ghost/settings...` as `/ghost/#/settings/...`, this updates the admin redirect on successful magic link validation to directly use the latter to avoid extra redirect
no issue
- adds a set of hardcoded "content image sizes" to the base config
- adjusts `handle-image-sizes` middleware to always allow the hardcoded content image sizes to be genreated
- updates `@tryghost/kg-card-factory` to allow passthrough of options to card renderers
- updates `@tryghost/kg-default-cards` to add `srcset` output for image and gallery cards
closes#11008
- Updated @nexes/nql to 0.4.0
This version exports the mapKeyValues utility function
- Replaced nql-map-key-values with @nexes/nql util fn
Usage was found using `rg nql-map-key-values` and replaced globally.
- Deleted nql-map-key-values module in shared
Now that this module isn't referenced anywhere else, we can remove it,
relying solely on the util exported by @nexes/nql
refs https://github.com/TryGhost/members.js/issues/43
- Adds new `isStripeConfigured` flag to public members site data which denotes if stripe setup is completed
- Helps clients like members.js/themes to configure payment behavior based on this flag
no-issue
The service at stripe.ghost.org must know which client_secret to use,
either the test, or live one. By encoding a JSON object as the state we
are able to pass data through the flow to inform this decision at the
end.
Note, that we still keep a random value in the state to protect against
CSRF attacks.
no-issue
We've added a "mode" query param to the members_stripe_connect api auth
method, allowing the client to easily switch between live and test mode.
no-issue
This adds the ability to pass a `mode` param of 'test' to the members stripe
connect service, which will ensure we use a testmode client_id for the
Stripe Connect OAuth flow.
This will allow users to connect their account in testmode.
no issue
- When imported member contains stripe_customer_id data but there is no Stripe configured on the Ghost instance such import should faiil. The logic is consistent with one where import fails after not being able to find customer in linked Stripe account
- Fixed import stats to show import failures instead of "duplicate" when the validation error is of "Stripe" origin
no issue
- The aim was to extract format-csv to become an external dependency. After some analysis found out that native papaparse method `unparse` was achieving the same results with a lot better test coverage and stability. Because papaparse will become Ghost's default csv processor in near future decided to integrate papaparse instead of extracting module for the format-csv module's code, which would become redundant soon anyways.
- For reference papaparse will substitute current csv-parser lib because it's better performance and maturity.
- Performance comparison can be checked here - https://github.com/Keyang/csvbench#result . At the time of writing papaparse is rougly 40% faster than csv-parser
refs https://github.com/TryGhost/Ghost/pull/11807
- this was originally reported in the PR above, but we could not
reproduce it on master
- presumably the user had the latest version of moment installed for
other purposes and so they were seeing the issue
- between moment 2.24.0 and 2.26.0, something must have changed which
stopped the previous functionality working
refs https://github.com/TryGhost/Ghost/issues/11414
Confirms if the fromAddress for sending member emails is valid and accessible using magic link flow, allowing owners to update full from address including domain change.
- Extends member service to handle magic link generation and validation for email update
- Updates existing setting endpoint to not directly update from address
- Adds new endpoint to send magic link to new address
- Adds new endpoint for validating the magic link when clicked and update the new email for from address
- Adds new email template for from address update email
refs https://github.com/TryGhost/Ghost/issues/11414
We want to allow adding custom domains to member's from address as long as the new email is verified using the magic link flow and not saved directly to DB. Since we currently don't store domain as part of fromAddress, this PR -
- adds migration to update all existing fromAddress by appending site domain
closes https://github.com/TryGhost/Ghost/issues/11768
- Wraps from parameter in double quotes so mail clients can read it whole
- Escapes double quotes in site title to avoid clash with wrapping
- Currently theme uploads delete the existing theme before copying the new files into place
- If something goes wrong with the delete action, you will end up in a bad state
- Some or all of the files may be deleted, but now Ghost won't try to put the new theme in place, instead returning an error
- This leaves you with an invalid active theme and a broken site
- Unlike delete, move is a one-hit operation that succeeds or fails, there moving a theme is safer than deleting
- This updated code moves the old theme to a folder with the name [theme-name]-[uuid] before copying the new theme into place
- Even if this fails, the files should not be gone
- There's a cleanup operation to remove the theme backup at the end, but we don't care too much if this fails
refs https://github.com/TryGhost/Ghost/issues/11212
- if a bookmark card fetch is performed (either directly or from fallback) and the page does not have an extractable title, return a more specific error message than "No provider found for supplied URL."
no issue
- When the customer cannot be imported because they are missing from linked Stripe account or the linked account is incorrect one, these new messages should provide a better clue about what has caused the error and how to act on it.
no issue
- There is a need to be able to label certain import group of members
with custom labels. This will allow to distinguish/filter these newly
imported members.
- Allowed `POST /members/csv/` endpoint to accept `labels`
field parameter which assigns labels to every member from imported csv.
no issue
- Similarly to other additive api methods (e.g. members.add) returned more specific ValidationError with contex filled in with the reason why adding did not succed.
- This change is needed for more graceful label handling when adding new members through import
no issue
- Adding labels doesn't cause any content to invalidate, similarly to adding members. Unlike it's caunterpart - tags, there is no dependent "frontend" content that would become invalid
no-issue
This was because the API was recieving a default `null` value for the
stripe_connect_integration_token setting.
This also improves the logging for this error.
no issue
- added an `externalRequest` lib
- uses same underlying `got` module as our `request` lib
- uses `got`'s `beforeRequest` and `beforeRedirect` hooks to perform it's own dns resolution for each url that's encountered and aborts with an error if it resolves to a private IP address block
- includes a bypass for Ghost's configured url so that requests to it's own hostname+port are not blocked
- updated v2 and canary oembed controllers to use the `externalRequest` lib
no-issue
If the stripeDirect config value is NOT set / false, then connect token
should be used over the API keys if both are present, or else use
whichever is present. If the stripeDirect config value is set /
true,then API keys should be used and stripe connect token should be
completely ignored.
The reason for this is that we want to use Stripe Connect as the primary
auth method going forward, but we want to keep the direct version
availiable should the Ghost Foundation ever dismantle. This will allow
people to use all the same features with no dependency on an external
service.
no-issue
In order to issue a redirect we need access to the "raw" req/res
objects, which is why we must return the function which gets access to
them.
The members service is used to create the auth url and to update the
users session.
no-issue
Uses the members service to parse a stripe_connect_integration_token
setting and set the stripe_connect_integration based on that.
This change includes ignoring the stripe_connect_integration{,_token}
settings, as the token is never saved, and the integration can only be
set by using the token.
no-issue
This module handles the creation of a url used for authorization of
Stripe Connect, and also the parsing of the data eventually received
from the authorization flow.
no-issue
There was some unused code here, the variable was never used, also we
were looping and collecting a list of errors, but only every using the
first one, so switched to the `find` method which stops iteration after
an element has matched.
closes https://github.com/TryGhost/Ghost/issues/11841
- Migration adds mapping between permissions and roles for email_preview send test mail
- Only owner previously had correct permission to send test emails
- Fixture existed to allow Admin/Editor/Integrations to send test mails but had missing migration
- Adds tests for roles to send test email
refs https://github.com/TryGhost/members.js/issues/30
- Added new `updateEmail` type for sending email address update confirmation mail to member
- The link in email updates member's email address
no issue
- in the near future, non-Ghost Ignition type errors will be coming into
Sentry
- because they don't have a statusCode, they'll be rejected
- we want to detect if they're non-Ghost and still deal with them
no issue
- node was complaining the package was missing after the sanitize-html update [1]
- the update removed an old version of lodash which supported the dot importing
method
- our code relied on this subdependency to work, but the structure has since
been updated
[1]: https://github.com/TryGhost/Ghost/pull/11867
- Ghost Admin has a redirect system built in, so not using the # does work
- However, the usage of # or no # was inconsistent, which is always wrong
- Using the correct Ghost Admin URLs makes it easier to reason about the behaviour
closes#11762
- iframe embeds which include a line break were undetected by the regex which checks for the existence and injects the required amp script
- this fix updates the regex to include any non-word character, as line-breaks are not included in the general `.` token
- Represents that logging is shared across all parts of Ghost at present
* moved core/server/lib/common/logging to core/shared/logging
* updated logging path for generic imports
* updated migration and schema imports of logging
* updated tests and index logging import
* 🔥 removed logging from common module
* fixed tests
refs https://github.com/TryGhost/members.js/issues/36
- Replaces existing members.js with new script which covers existing functionality as well as adds new ones
- This is behind developer experiments flag atm
refs https://github.com/TryGhost/members.js/issues/30
- Member cannot update their email directly but need to do it via magic link sent to new email address
- Previous profile update change had allowed email to be updated directly as well for authenticated member
no issue
- adds `search` bookshelf plugin that calls out to an optional `searchQuery()` method on individual models to apply model-specific SQL conditions to queries
- updated the base model's `findPage()` method to use the search plugin within `findPage` calls
- added a `searchQuery` method to the `member` model that performs a basic `LIKE %query%` for both `name` and `email` columns
- allowed the `?search=` parameter to pass through in the `options` object for member browse requests
* moved `server/config` to `shared/config`
* updated config import paths in server to use shared
* updated config import paths in frontend to use shared
* updated config import paths in test to use shared
* updated config import paths in root to use shared
* trigger regression tests
* of course the rebase broke tests
* refactored core/frontend/services/proxy to import common dependency like a normal person
* removed all imports of `common/errors`
* 🔥 removed common/errors module
Co-authored-by: Vikas Potluri <vikaspotluri123.github@gmail.com>
no issue
- moves members stats generation for the admin graph from the client to the server
- outputs a basic totals count across a requested date range of 30, 90, 365 days, or all time. See below for the response shape
- leaves heavy lifting of the counts to the SQL engines - tested on a dataset of 100k members and query performance is <100ms
```
GET /ghost/api/canary/members/stats/?days=30
{
total: 100000,
total_in_range: 20000,
total_on_date: {
'2020-04-25': 19000,
'2020-04-26': 19500,
// continues until today's date
},
new_today: 200
}
```
* refactored `core/frontend/apps` to destructure common imports
* refactored `core/frontend/services/{apps, redirects, routing}` to destructure common imports
* refactored `core/frontend/services/settings` to destructure common imports
* refactored remaining `core/frontend/services` to destructure common imports
* refactored `core/server/adapters` to destructure common imports
* refactored `core/server/data/{db, exporter, schema, validation}` to destructure common imports
* refactored `core/server/data/importer` to destructure common imports
* refactored `core/server/models/{base, plugins, relations}` to destructure common imports
* refactored remaining `core/server/models` to destructure common imports
* refactored `core/server/api/canary/utils/serializers/output` to destructure common imports
* refactored remaining `core/server/api/canary/utils` to destructure common imports
* refactored remaining `core/server/api/canary` to destructure common imports
* refactored `core/server/api/shared` to destructure common imports
* refactored `core/server/api/v2/utils` to destructure common imports
* refactored remaining `core/server/api/v2` to destructure common imports
* refactored `core/frontend/meta` to destructure common imports
* fixed some tests referencing `common.errors` instead of `@tryghost/errors`
- Not all of them need to be updated; only updating the ones that are
causing failures
* fixed errors import being shadowed by local scope
no issue
- Removes global bodyParser middleware for membersApp and adds it to specific endpoints
- Removes global boolParser middleware for membersApp
We added bodayParser middleware to memebrsApp in [this](fe3eab1836) commit to read json requests for members update endpoint, but that had issues with stripe webhook parsing for `/webhooks` endpoint as stripe expects raw data to be passed down.
no issue
- updated `filter` plugin with appropriate label relationship and replacement config for NQL
- fleshed out member and label fixture data and tools to facilitate tests
no-issue
The flag currently defaults to `true` as we are still using stripe
direct. We expose it on the admin api config endpoint so that the
Ghost-Admin client can use it to conditionally render
- Allows member logged in with valid session to update their profile info - name, email, subscribed(newsletter subscription status)
- Adds new util method for formatted member response on the endpoints
- Adds common middlewares for body/bool parser and maintenance
- Adds `subscribed` status to member response
no issue
- shows the full URL instead of a truncated link in password reset email body. This is required for security and usability reasons (copy/paste)
