refs https://github.com/TryGhost/Team/issues/757
- There is no usecase for editing "labs" settings outside of canary/v4 API versions. Removing support for older versions makes the supported API surface smaller (easy maintenance).
refs https://github.com/TryGhost/Team/issues/757
- To safeguard from mise of a very permissing "object" value of the "labs" setting this change introduces an "allowlist" approach to filtering unrecognized labs flags
- Should allow maintainers to have a clear view of which labs flags are currently in use and manage them accordingly
no refs
The product output serializer is removing the include data due to the includes being missing in frame options for some reason. This is a temporary fix that always allows the default includes as `monthly/yearly_price` to unblock the API, and we can revert it back to explicit request once fixed.
* 🐛 Fixed saving Members with Complimentary plans
refs https://github.com/TryGhost/Team/issues/758
Since 4.6 The Admin is using the comped flag again, rather than creating
subscriptions for zero-amount prices directly. With the `comped` flag
removed, the default state was for it to be falsy in the Admin, and when
saved would trigger the legacy comped flow, cancelling the subscription.
This reverts commit 57a176ff3d.
refs https://github.com/TryGhost/Team/issues/712
- Adds a Content API for products, which can be used by the theme-engine
middleware to populate the products data.
- Removes Stripe ids from Content API so they cannot be used to
initiate checkout sessions directly
- The monthly_price and yearly_price are used to create new prices, and
to set them to the default monthly & yearly price for the product.
refs https://github.com/TryGhost/Team/issues/754
- This is fixing the root cause of an error being saved in `settings` table under `notifications` key. There needs to be a follow up to this fixing any possible instances that might have been affected byt the bug
no issue
- moved `config` and `site` API output generation to a `public-config` service allowing all API versions to use `publicConfig.config` or `publicConfig.site` in their query methods
- updated `config` and `site` API output serializers to use an allow-list that limits the data returned for each API version
refs https://github.com/TryGhost/Team/issues/598
Stripe Webhooks require SSL in production, and so we should not be
allowing connecting to Stripe in production mode unless the site is
running with SSL.
refs https://github.com/TryGhost/Team/issues/693
Since we no longer have the concept the "comped" we update the v3 API to
always have a `comped` flag of `false` - maintaining backwards
compatibility.
refs a4c78dbf19
Updates member data on edit to include products data when comped status is changed, as by default we don't include products data when member goes from free to paid subscription due to comped being added.
closes https://github.com/TryGhost/Team/issues/699
With custom products, saving a member with subscriptions on member detail page in Admin throws errors on console, though the save is successful. This breaks the Admin as user needs to refresh the screen again to get rid of error. This change -
- updates the response on member save to return `price` object in subscription
- updates tests
refs https://github.com/TryGhost/Team/issues/581
closes https://github.com/TryGhost/Team/issues/582
Emails can now be sent to members with specific associated labels or products by specifying an NQL string. We want to bring the same members segment feature to content by allowing `visibility` to be an NQL filter string on top of the `public/members/paid` special-case strings.
As an example it's possible to set `posts.visibility` to `label:vip` to make a post available only to those members with the `vip` label.
- removed enum validations for `visibility` so it now accepts any string or `null`
- bumped `@tryghost/admin-api-schema` for API-level validation changes
- added nql validation to API input validators by running the visibility query against the members model
- added transform of NQL to special-case visibility values when saving post model
- ensures there's a single way of representing "members" and "paid" where NQL gives multiple ways of representing the same segment
- useful for keeping theme-level checks such as `{{#has visibility="paid"}}` working as expected
- updated content-gating to parse nql from post's visibility and use it to query the currently logged in member to see if there's a match
- bumped @tryghost/members-api to include label and product data when loading member
refs https://github.com/TryGhost/Team/issues/667
On clean and existing installs, the default product created should be named the same as the site title instead of the name in fixture. This change updates the default product's name to site title during the site setup. We use the Product name in Portal.
refs https://github.com/TryGhost/Team/issues/581
refs https://github.com/TryGhost/Team/issues/582
When publishing a post via the API it was possible to send it using `?email_recipient_filter=all/free/paid` which allowed you to send to members only based on their payment status which is quite limiting for some sites.
This PR updates the `?email_recipient_filter` query param to support Ghost's `?filter` param syntax which enables more specific recipient lists, eg:
`?email_recipient_filter=status:free` = free members only
`?email_recipient_filter=status:paid` = paid members only
`?email_recipient_filter=label:vip` = members that have the `vip` label attached
`?email_recipient_filter=status:paid,label:vip` = paid members and members that have the `vip` label attached
The older `free/paid` values are still supported by the API for backwards compatibility.
- updates `Post` and `Email` models to transform legacy `free` and `paid` values to their NQL equivalents on read/write
- lets us not worry about supporting legacy values elsewhere in the code
- cleanup migration to transform all rows slated for 5.0
- removes schema and API `isIn` validations for recipient filters so allow free-form filters
- updates posts API input serializers to transform `free` and `paid` values in the `?email_recipient_filter` param to their NQL equivalents for backwards compatibility
- updates Post API controllers `edit` methods to run a query using the supplied filter to verify that it's valid
- updates `mega` service to use the filter directly when selecting recipients
refs https://github.com/TryGhost/Team/issues/496
We want to give more control over the default selection of email recipients when publishing a post, to do that we need somewhere to store those settings. These settings are site-wide and intended for use by admins to control the default editor behaviour for all staff users. They _do not_ control API behaviour, if you want to send email when publishing via the API it's still necessary to explicitly opt in to that using the `?email_recipients_filter=` query param.
- new `editor` settings group to indicate that these settings only affect the UI rather than the API
- `editor_default_email_recipients` controls overall behaviour, string/enum with these allowed values:
- `'disabled'`: no option to send email is shown in the editor's publishing dropdown
- `'visibility'`: (default) selected member segment is dynamic and matches the post visibility filter
- `'filter'`: specific member filter defined in `editor_default_email_recipients_filter` setting
- `editor_default_email_recipients_filter` is an NQL string for selecting members, used when `editor_default_email_recipients` is set to `'filter'`
- default value is `'all'`
- the segment string can be any valid NQL filter with the additional special-case values of `'all'` and `'none'`
refs 829e8ed010
- i18n is used everywhere but only requires shared or external packages, therefore it's a good candidate for living in shared
- this reduces invalid requires across frontend and server, and lets us use it everywhere until we come up with a better option
- Having these as destructured from the same package is hindering refactoring now
- Events should really only ever be used server-side
- i18n should be a shared module for now so it can be used everywhere until we figure out something better
- Having them seperate also allows us to lint them properly
refs: bf0823c9a2
refs: ae86254972
- continuing the work of splitting up the theme service into logical components
Themes Service
- The serverside theme service now serves just the API and boot
- It loads the theme and passes it to the theme-engine via the bridge
This achieves the bare minimum goal of removing all the cross requires between server and frontend around themes
There is still a lot more to do to achieve an ideal architecture here as laid out in ae86254972
refs https://github.com/TryGhost/Team/issues/616
We need a way to assign Products to Members via a Subscription, and we've
followed the same pattern as the editSubscription method for the Members API
controller, which acts upon Subscriptions as a nested resource.
Subscriptions now are linked to products, and we've included those links by
default in the Member Admin API as we already include subscriptions by
default, and Products are now a core part of the Members feature-set.
refs https://github.com/TryGhost/Team/issues/616
This is a basic scaffold of the API to get things moving.
The input serializer is so that the controller logic does not need to
know about the json-api shape of the input data.
The output serializer is an adaptation of the members one.
refs https://github.com/TryGhost/Team/issues/618
- The `oauth_client_id` and `oauth_client_secret` are placeholders to store OAuths related data.
- The flag for `oauth_enabled` or anything along those lines was not added intentionally in favour of checking if the `oauth_client_id` & `oauth_client_secret` are null.
refs https://github.com/TryGhost/Team/issues/585
- adds `DELETE /members/` route to the Admin API
- supports `?filter`, and `?search` query params to limit the members that are deleted
- `?all=true` is required if no other filter or query is provided
- uses `models.Member.bulkDestroy` which _will not_ cancel any Stripe subscriptions if members have them but _will_ clean up the Stripe relationship data in Ghost's database
no refs
- Removes old `/members/stats` endpoint in favor of new `/members/stats/count` in canary/v4 which captures members counts using new events table
- Removes tests for old `/members/stats` endpoint
- Added test for new `/members/stats/count` endpoint
refs https://github.com/TryGhost/Team/issues/555
- Previous blocklist approach was resulting in adding every single new table into an export automatically. Which creates possibility to leak sensitive data if not used porperly. Allowlist approach gives better control over what is exported, makes this information explicit, and version-control friendlier
closes https://github.com/TryGhost/Team/issues/552
Refactors URL transforms so they take place at the model layer rather than the API serializer layer. Continuation of the pattern created for the settings model in https://github.com/TryGhost/Ghost/pull/12738
- Added checks to all front-end tests to ensure output does not contain the magic replacement string
- includes failing acceptance test for `__GHOST_URL__` appearing in sitemaps
- Removed all transform-ready URL transforms from API serializers
- input serializers transform image urls relative->absolute to keep absolute-urls as the consistent "outside of the database" format
- output serializers should not need to perform any URL transforms as that will be done at the model layer
- Added url transforms to models layer
- removes knowledge from the API serializers which shouldn't need to know how data is stored internally in the database
- makes absolute urls the consistent "outside of the database" URL format
- adds transform step to the sitemap generator because the data used for that is fetched directly via knex which will not run through the bookshelf `parse()` methods
refs 6b07d4b2a0
- The model is needed here, because it contains full set of fields. In some cases, like email-preview, the "plaintext" field is not present in "attrs" which causes the logic to fail.
- This should be sorted along with https://github.com/TryGhost/Ghost/issues/10396
refs https://github.com/TryGhost/Team/issues/467
refs a6f5eb71be
- When a generated excerpt is calculated for posts/page resources it uses raw model! to get the data. Model contains untranformed __GHOST_URL__ markup which has to be additionally processed before extracint an excerpt or use the transformed `plaintext` from available attributes (chose the latter to decrease complexity)
- Removed model dependency as `attrs` at this point of serialization should always contain the `plaintext` field. It's ugly and has an unsolved bug report here - https://github.com/TryGhost/Ghost/issues/10396. The reliance should be solved at some point, but definitely not a part of this issue
refs https://github.com/TryGhost/Ghost/pull/12736
refs https://github.com/TryGhost/Team/issues/467
knex's `parse()` method is only called on data when directly fetched from the db. This was causing problems when model instances are passed around via events for example because `.get('key')` will return data that was directly set on the model without having gone through the `parse()` transformations. The result of this inconsistency was settings appearing correct when Ghost started up but then being broken as soon as a setting was changed.
- moved absolute/relative->transform-ready URL transformations from the API input serializers to the model's `format()` method and replaced with a relative->absolute transform in API input serializers
- results in consistency because `.get()` on a settings model will always return an URL
- removed transform-ready->absolute transforms from the API output serializers as that is now handled at the model-layer
closes https://github.com/TryGhost/Team/issues/467
- switches to storing "transform-ready" URLs in the database
- transform-ready URLs contain a `__GHOST_URL__` placeholder that corresponds to the configured url that gives a few benefits
- much faster and less memory intensive output transformations through not needing to parse html or markdown - the transform can be achieved using a straightforward regex find+replace
- ability to change to/from or rename subdirectory without any manual updates to the database
- modified existing 4.0 url-transformation migration rather than adding another one and repeating the transformation on posts rows
refs https://github.com/TryGhost/Team/issues/469
- Cleans up response format for mrr and volume stats endpoint to more consistent pattern
- Removes `unit` attribute for now as its not used
refs: https://github.com/TryGhost/Team/issues/510
- Changed to return the full hostSettings key, not just the billing URL
- We are introducing several more settings that are needed by Admin including limits
- Passing the whole object makes this much easier to reason about as Admin has the exact same config as the server
refs: https://github.com/TryGhost/Team/issues/510
- The current member limit was implemented as a member-specific concept
- The new limit service is much more generic, here we are swapping old for new
- The updated concept here is blocking all publishing, not just email sending, when a site is over its member limit
- To determine that we are publishing a post, we must be in the model layer. The code has been moved to the permissible function which makes sense as this is a permissions error that we are throwing
- I've left the extra check for email retries in, in case there is some loophole here (but we may wish to change it)
refs https://github.com/TryGhost/Team/issues/510
- When the host config was introduced it was incorrectly introduced as host_settings instead of hostSettings
- All other Ghost config uses camelCase, so changing this now before it becomes a problem
- Note: Also removed some rogue return awaits that don't make sense. It's not possible to hit that case, but cleaning up anyway
refs https://github.com/TryGhost/Team/issues/509
- Allows to update and read 'locale' key along with the deprecated 'lang'
- In Ghost v5 the 'lang' key will be dropped and the migration in settings table will clean up the key name to match the one exposed through the APIs
refs https://github.com/TryGhost/Team/issues/513
- Having exports in no particular order was making it hard to spot if there's anything missing
- Having a DESC order on the exported API versions makes it very easy to modify and maintain in the future
refs https://github.com/TryGhost/Ghost/issues/12716
- The code in serializePostModel was broken and always defaulted to 'v3'! It refered to non-existent `model.get('api_version')` there's no such field in posts model! Changed the implementation so that the API version is passed in as a parameter to the method instead
- The style of providing "defaults" everywhere creates a need for future maintenance when we bump the version e.g in Ghost v5. Maybe reworking these methods to require a passed version and throwing an error instead would be more maintainable long-term?
refs https://github.com/TryGhost/Team/issues/469
The initial implementation was just to get us going with the dashboard
but the requirements have changed now, rather than updating the code we
allow to pass the `limit` options so the Admin can choose how many
events to display.
refs https://github.com/TryGhost/Ghost/issues/12633
Adds new `browse` endpoint for emails that allows Admin to check performance of newsletters over time and show stats on dashboard as primary usecase
closes https://github.com/TryGhost/Team/issues/468
- updated post-gating
- clears excerpt if there's no access
- rebuilds excerpt from free preview if paywall card is used and there's no custom excerpt
closes https://github.com/TryGhost/Team/issues/466
- upgraded kg-default-cards to include paywall card
- extracted `htmlToPlaintext` from post model to shared util for re-use
- updated post-gating to set html+plaintext to the free preview if a paywall card has been used
- re-generates plaintext from the truncated html using `htmlToPlaintext` util
- display free content in the `{{content}}` helper via the default CTA template
refs https://github.com/TryGhost/Ghost/issues/10318
- API changes introduced:
canary/v4 Admin API
GET /settings/ (browse)
+ "unsplash" present in response as boolean value
GET /settings/:settingName (read)
+ "unsplash" present in response as boolean value
PUT /settings/ (edit)
+ "unsplash" updates setting, accepts ONLY boolean format
v3 Admin API
GET /settings/ (browse)
+ "unsplash" present in response with object value
GET /settings/:settingName (read)
+ "unsplash" present in response with object value
PUT /settings/ (edit)
+ "unsplash" updates setting, accepts either boolean or object formats
v2 Admin API
GET /settings/ (browse)
+ "unsplash" present in response with object value
GET /settings/:settingName (read)
+ "unsplash" present in response with object value
PUT /settings/ (edit)
+ "unsplash" updates setting, accepts object format
refs https://github.com/TryGhost/Ghost/issues/10318
- `labs` setting is dropped from setting values as the use of JSON objec
to sore settings has been deprecated
- `labs` setting is no longer accepted as a paramter in the Settings API nor the
impoprter. The value is ignored if present in the POST/PUT requests and
returns 404 in case it is requested by key at `GET /settings/:key`
refs https://github.com/TryGhost/Ghost/issues/12633
Adds new stats endpoints for showing member dashboard, which fetches aggregate date over member events tables to form relevant datasets -
- Adds new endpoint for tracking MRR of site over time - `/members/stats/mrr`
- Adds new endpoint for tracking subscribers on site over time - `/members/stats/subscribers`
- Adds new endpoint for tracking gross volume of site over time - `/members/stats/gross_volume`
- Adds new endpoint for tracking member count on site over time - `/members/stats/count`
refs https://github.com/TryGhost/Ghost/issues/12608
- adds `admin/canary/themes/install` endpoint to the Admin API
- requires two query params. `source` must be set to "github". `ref` should refer to a GitHub repo in the format "{org}/{repo}"
- downloads zip archive for the repo from github
- runs downloaded zip through the same process as uploaded zips
closes https://github.com/TryGhost/Ghost/issues/12600
The bookshelf-relations plugin which we use will **remove** all
relations when they are set to an empty array, but will leave them alone
if it's set to undefined.
Our logic to deduplicate uppercase & lowercase version of the same label
was in advertently always setting the labels to an array, but when the
model was saved without passing the labels, this array would be empty.
Here we've added a check which will skip all label handling, if there
are no labels set.
no-issue
This removes all references to the members labs setting, any code that was run conditionally behind this flag now runs unconditionally.
* Removed usage of Members labs flag
* Removed tests for Members disabled
* Added dynamic keypair generation for when setting is missing
no refs
- Updates member model serializer to directly set subscriptions on member object instead of `stripe.subscriptions`
- Updates all references to members subscriptions from nested `stripe.subscriptions` to `subscriptions`
- Updates v3 API serializer to still use `stripe.subscriptions`
- Updates tests
no-issue
* Removed support for paid param from v3 & canary API
* Updated active subscription checks to use status flag
* Updated MEGA to use status filter over paid flag
* Removed support for paid option at model level
* Installed @tryghost/members-api@1.0.0-rc.0
* Updated members fixtures
refs #12160
This flag will allow us easier filtering of members via the API
* Added status column to members table
This flag will be used to determine if a member is free or paid, rather
than relying on joins with the customers and subscriptions tables.
* Added migration to populate members.status
As we add the column with a default value of "free" we only need to care
about the paid members here. We also preemptively handle migrations for
SQLite where there are > 998 paid members.
no refs
Adds new FirstPromoter settings similar to amp, which allows sites to take advantage of FirstPromoter to launch their own member referral program natively.
- Adds new firstpromoter settings group
- Adds `firstpromoter` setting to group
- Adds `firstpromoter_id` setting to group for FirstPromoter referral tracking id
- Updated tests
refs #12537
- Refactored overuse of rewire mocking blog-version.
- Fixed a bug introduced along the way when duplicate notifications errored instead of returning empty result
refs #12537
- notifications controllers were overbloated with non controller related code and were identical. It is important to reduce unnecessary code ahead of v4 API introduction
- Follow up commit will transform newly created module into a class following DI pattern
refs #12537
- Class syntax is preferred over module functions because of constructor parameter injection (DI) which allows for easier module decoupling and testing
refs #12537
- `stats` method in members controller is quite big and does much more then controller method code should - few calls to relevant modules
- Extracted code "as is" into members serivce
- Next step will be to refactor this module as a class pattern with DI parameters
closes#12503
Since we include _all_ subscriptions in the `stripeSubscriptions`
relation, we must check that for an existing active complimentary
subscription to decide whether or not we should skip creation of a
complimentary subscription
no-issue
We now include all subscriptions as part of the member, so we need to
ensure the comped flag is only true if the member has an **active**
complimentary plan
refs https://github.com/TryGhost/Ghost-Admin/pull/1796
We want to be able to display an email activity timeline in Ghost-Admin for each member. The quickest way to achieve that right now is to provide access to the `email_recipient` data for the member when fetching, this will allow clients to build up a timeline based on the event timestamps included with each email_recipient/email pair.
- sets up `email_recipients` relationship in `Member` model
- updates members API read endpoint to accept an `email_recipients` include parameter
- appends `email_recipients.email` to the `withRelated` array when `email_recipients` is included so that we have data available for email subject and html/plaintext for previews
- updates members API output serializer to include the email_recipients object in the output
refs https://github.com/TryGhost/Ghost/issues/12461
- adds `members.email_count` and `members.email_opened_count` columns to contain cached counts for faster queries when outputting member data via API
- adds migration to populate cached counts with existing data
- tested locally on ~50k members which took ~4sec on mysql
- updates members output serializer to include the new fields in API output
refs https://github.com/TryGhost/Ghost/issues/12421
- nullable so we can distinguish between members that have and haven't received any trackable emails
- indexed because we'll be using this column for sorting
no issue
- email analytics may be desirable to fully switch off in certain circumstances, when that happens we want to prevent related background jobs from running and expose the feature flag via the config endpoint in the Admin API so that clients can adjust accordingly
closes#12083
- fixes a parsing issue where negative offset values were incorrectly having the + sign added regardless of actual offset for sqlite databases.
- for mysql databases absolute values of offset were taken with sign applied where appropriate to stop issues where both hours and minutes could be negative which would cause both an issue with offsets that could present as -2:30 and by the look of the code also trigger extra padding to result in -2:-030 rather than the expected -2:30
closes https://github.com/TryGhost/Ghost/issues/12416
This fixes compatibility for the `send_email_when_published` option for
the Posts API.
The model layer only allows setting the `email_recipient_filter` column
when the `status` is being changed. Because of this we need to withhold
the `status` change until after we have determined the
`email_recipient_filter`.
refs https://github.com/TryGhost/Ghost/issues/10354
- The serializer wasn't hooked up properly during the implementation (1a4497fc9a). It is not possible to hook it back in now as that would introduce breaking change to now stable v2 API.
- v3 (canary) has is properly hooked in, so there should not be a problem going forward
no-issue
* Handled send_email_when_published in Posts API
This restores backwards compatibility of the Posts API allowing existing
clients to continue to use the `send_email_when_published` flag. This
change uses two edits, which is unfortunate. The reason being is that
this is an API compatibility issue, not a model issue, so we shouldn't
introduce code to the model layer to handle it. The visibility property
of the model is used to determine how to fall back, and because it can
be left out of the API request, and relies on a default in the settings,
we require that the model decide on the `visibility` before we run our
fallback logic (or we duplicate the `visibility` default at the cost of
maintenance in the future)
* Dropped send_email_when_published column from posts
Since this column is not used any more, we can drop it from the table.
We include an extra migration to repopulate the column in the event of
a rollback
* Updated importer to handle send_email_when_published
Because we currently export this value from Ghost, we should correctly
import it. This follows the same logic as the migrations for this value.
* Included send_email_when_published in API response
As our v3 API documentation includes `send_email_when_published` we must
retain backward compatibility by calculating the property.
* Fixed fields filter with send_email_when_published
* Added safety checks to frame properties
Some parts of the code pass a manually created "frame" which is missing
lots of properties, so we check for the existence of all of them before
using them.
* Fixed 3.1 migration to include columnDefinition
We require that migrations have all the information they need contained
within them as they run in an unknown state of the codebase, which could
be from the commit they are introduced, to any future commit. In this
case the column definition is removed from the schema in 3.38 and the
migration would fail when run in this version or later.
refs https://github.com/TryGhost/Ghost/issues/12355
- Adds new default settings for newsletter customisations - `newsletter_show_badge`, `newsletter_show_header` and `newsletter_body_font_category`
- Adds migrations to update group for new settings
- Add migration to update settings based on existing config value for newsletter settings
- Passes new newsletter settings to newsletter template and updates design based on them
- Fix tests
no-issue
* Used email_recipient_filter in MEGA
This officially decouples the newsletter recipients from the post
visibility allowing us to send emails to free members only
* Supported enum for send_email_when_published in model
This allows us to migrate from the previously used boolean to an enum
when we eventually rename the email_recipient_filter column to
send_email_when_published
* Updated the posts API to handle email_recipient_filter
We now no longer rely on the send_email_when_published property to send
newsletters, meaning we can remove the column and start cleaning up the
new columns name
* Handled draft status changes when emails not sent
We want to reset any concept of sending an email when a post is
transition to the draft status, if and only if, and email has not
already been sent. If an email has been sent, we should leave the email
related fields as they were.
* Removed send_email_when_published from add method
This is not supported at the model layer
* Removed email_recipient_filter from v2&Content API
This should not be exposed on previous api versions, or publicly
* Removed reference to send_email_when_published
This allows us to move completely to the email_recipient_filter
property, keeping the code clean and allowing us to delete the
send_email_when_published column in the database. We plan to then
migrate _back_ to the send_email_when_published name at both the
database and api level.
- 504509bb6 removed the global override for Promise
- there are a bunch of places in code that use Bluebird Promise methods,
but Bluebird wasn't being imported in these places
- this would have thrown errors all over the place
closes#11085
- Ghost has been using YAML format for other configurations (e.g. routes). The plan is to move to this format for all user-edited settings files. By default JSON format is still used in Ghost Admin API v2/v3, but will be changed to YAML in API v4. Check referenced issue for more context.
- New format supports all the features available before. The main noticeable change is the structure of config file. It is now grouped by redirect HTTP code instead of specifying `"permanent": true | false` attribute for each config property. Example format for YAML config:
```
302:
/from-url/: /to-url/
301:
/category/([a-z0-9\-]+)/i: /tag/$1/
/v([0-9\.]+)/docs/([a-z0-9\-]+)/i: /docs/$2/
```
- Added 2 new endpoints: `POST redirects/upload` and `GET redirects/download`. These serve as an alias to current GET/POST `/redirects/json. "upload/download" naming pattern is introduced to match the convention with other resources that can be uploaded and downloaded (images, themes etc.). `/redirects/json` endpoints will be removed in Admin API v4
- The parsing code from `custom-redirects.js` has been moved to `frontend/services/redirects/settings.js`. This location is more appropriate for this logic and eventually `custom-redirects.js` middlewear might be moved into "frontend" as this middlewear plays a role mostly effecting that area.
no refs
[Portal](https://github.com/TryGhost/Portal) is a new drop-in script to make the bulk of Ghost membership features work on any theme out of the box, which was under a developer flag so far. This release removes the flag for Portal and makes it included as default for any members-enabled Ghost site. The Portal script is backward compatible with old public members script and existing Members-enabled themes should notice no change.
- Removes Portal config flag as Portal is now enabled by default
- Removes old members script as Portal is backward compatible with it
- Changes `{{content}}` helper to show default CTA in case of restricted content access
- `accent_color` setting is no more behind the dev experiment flag and included by default
- Adds migration to switch off Portal button setting for all existing sites which don't have Portal enabled in beta
no issue
- standard browse/read/add/edit/destroy API endpoints for snippets resource
- updates `@tryghost/admin-api-schema` dependency to version that includes snippet definition and schemas
closes#12273
- `comped` field has been allowed when editing a member or importing from a CSV. There has been a usecase (Zapier Integration) for API client to create a member with "Complimentary" plan, which made this change necessary
- Previously the logic for comped field was to skip and continue member record creation if Stripe was not connected. Now we throw an error - same as the one we have been throwing before when stripe_customer_id field was passed in. The implication of this change is that we won't be creating any record now if comped === true and Stripe is disabled.
- Bumped admin-api-schema-package. Contains `comped` schema change so this field gets passed through to controller
no issue
When scheduling a post to publish+send the "view online" link was pointing at https://site.com/404/ rather than the published post's url.
The problem occurred because the `/schedules/` endpoint wraps it's post read+edit calls in a transaction. Context:
- when a post is published with with the "send email" option the email record is immediately generated and added to the API response, as part of the email record generation we render the email content including fetching the url for the "view online" link
- urls for all resources are handled by our `url` service, that service updates it's internal cache based upon model events such as the "edited" event triggered when a post is published
- if the posts API controller is given a transaction, the email record is also generated inside of that transaction however at this point the `url` service will not have been updated because the post record hasn't been committed meaning it has no available url for the post
Fix:
- removed the `models.Base.transaction()` wrapper around the post read+update in the `/schedules/` API controllers
- we don't need a transaction here. It was added as protection against another write request coming in between the `/schedules/` controller reading a post and publishing a post but we already have protection against that in the form of collision detection - if a write request comes in and commits between the schedules controller reading the post and updating it, the scheduler's update call will fail with a collision error at which point the scheduler itself should retry the request which could then publish the post successfully if everything else is in order
closes https://github.com/TryGhost/Ghost/issues/12260
- if a card type was not explicitly chosen (i.e. a url was pasted into the editor) then abort fetching the oembed endpoint if we detect it's a `wp-json` oembed and return a bookmark card payload instead
- cleaned up an unused argument in the internal `fetchBookmarkData()` method
closes https://github.com/TryGhost/Ghost/issues/12247
- Internal preview controller was lacking "mapping" call to post object which handled not only missing meta attribute information but lots of other mappings (e.g. users, tags, etc.)
- Have added a regression test to catch issues like this in the future
no issue
- We had previously allowed accent_color setting for member site settings behind portal flag, but Ghost Admin also needs the public site setting with accent color to correctly reflect the accent color when flag is switched on
- Removes deletion of accent color setting when behind the Portal flag OR dev experiment flag
refs #12055
As part of the work in TryGhost/Members#206 we load the stripeCustomers relation on the member model, and we do not want this to be part of the API response. The changes here include a refactor but the main thing is that the serialized object is explicit and does not include unexpected or unknown fields.
* Moved mapMember out of mapper file
This cleans up the serializer a bit by keeping it's functionality all in
one place, rather than a shared mapper file
* Refactored members controller to return models
Previously the controller was calling toJSON, which is serialization,
this updates the controller to only deal with models, leaving all of the
serialization to the serializer!
* Refactored members serializer
This adds typings to all of the methods/functions in the serializer, as
well as making the serializating explicit, rather than returning the
result of toJSON, we explicitly set the properties we expect to be on
the output object. This protects us against accidental API changes in
the future.
closes https://github.com/TryGhost/Ghost/issues/10628
- JSON Schemas were extracted into a separate module to allow other clients to reuse them (for example documentation). Having them in a separate package also slims down the amount of code needed to be maintained in the core.
- Updated canary API input validators to use admin-api-schema module
- Removed canary schemas that moved into admin-api-schema package
- Updated v2 API input validators to use admin-api-schema package
- Removed v2 schemas that moved into admin-api-schema package
- Updated tests to contain needed information in apiConfig to pick up correct validation
- Added @tryghost/admin-api-schema package dependency
refs https://github.com/TryGhost/Ghost/pull/12232
When viewing sent emails in Ghost's admin area, it displays the `html` field directly from the `email` relation loaded with the post. Since the `mega` refactor we now store raw content in that field rather than sanitized "preview" content so it's necessary to modify the API output to match the old behaviour.
- use the API output serializers to parse replacements in email content and replace with the desired fallback or empty string
no issue
- store raw content in email record
- keep any replacement strings in the html/plaintext content so that it can be used when sending email rather than needing to re-serialize the post content which may have changed
- split post email serializer into separate serialization and replacement parsing functions
- serialization now returns any email content that is derived from the post content (subject/html/plaintext) rather than post content plus replacements
- `parseReplacements` has been split out so that it can be run against email content rather than a post, this allows mega and the email preview service to work with the stored email content
- move mailgun-specific functionality into the mailgun provider
- previously mailgun-specific behaviour was spread across the post email serializer, mega, and bulk-email service
- the per-batch `send` functionality was moved from the `bulk-email` service to the mailgun provider and updated to take email content, recipient info, and replacement info so that all mailgun-specific headers and replacement formatting can be handled in one place
- exposes the `BATCH_SIZE` constant because batch sizes are limited to what the provider allows
- `bulk-email` service split into three methods
- `send` responsible for taking email content and recipients, parsing replacement info from the email content and using that to collate a recipient data object, and finally coordinating the send via the mailgun provider. Usable directly for use-cases such as test emails
- `processEmail` takes an email ID, loads it and coordinates sending related batches concurrently
- `processEmailBatch` takes an email_batch ID, loads it along with associated email_recipient records and passes the data through to the `send` function, updating the batch status as it's processed
- `processEmail` and `processEmailBatch` take IDs rather than objects ready for future use by job-queues, it's best to keep job parameters as minimal as possible
- refactored `mega` service
- modified `getEmailData` to collate email content (from/reply-to/subject/html/plaintext) rather than being responsible for dealing with replacements and mailgun-specific replacement formats
- used for generating email content before storing in the email table, and when sending test emails
- from/reply-to calculation moved out of the post-email-serializer into mega and extracted into separate functions used by `getEmailData`
- `sendTestEmail` updated to generate `EmailRecipient`-like objects for each email address so that appropriate data can be supplied to the updated `bulk-email.send` method
- `sendEmailJob` updated to create `email_batches` and associated `email_recipients` records then hand over processing to the `bulk-email` service
- member row fetching extracted into a separate function and used by `createEmailBatches`
- moved updating of email status from `mega` to the `bulk-email` service, keeps concept of Successful/FailedBatch internal to the `bulk-email` service
no-issue
By using the "email" validation, we were validating emails in CSV
imports using a different validator to the rest of the API. AJV's built
in email validation was failing on emails with "special" characters,
such as letters with an umlaut above them.
This commit brings the validation for CSV imports in line with the rest
of the API.
refs #12033
- Allowing to change parent integration opens up possible security holes and has no clear usecase at the moment. After a webhook record is created it should not be possible to change parent integration.
- Had do partially duplicate JSON schema definition from webhooks definition as there is no proper composition technique available in current version of JSON Schema.
refs https://github.com/TryGhost/Ghost/issues/12033
refs https://github.com/TryGhost/Ghost/issues/10567
- Creating a webhook without valid parent integration leads to orphaned webhook records, which shoult not ever happen
- This scenario is only possible for non-integration authentication,
because in case of integration being authenticated it's id is
automatically assigned to creatd webhook
no-issue
refs: https://github.com/TryGhost/Members/commit/63942f03
The above commit updated all magic-link methods to be async, this change
ensures that we will handle a Promise return when it's updated in Ghost.
`await` has no effect on non Promise return values so it's safe to add
this now.
no issue
- The new Portal config flag allows switching on Portal conditionally with config
- The dev experiment flag still works for enabling Portal
- The flag currently defaults to `false` as Portal is still a beta feature and switched off by default
- We expose it on the admin api config endpoint so that the Ghost-Admin client can use it to conditionally render Portal settings
refs #2635
- Adds 'Location' header to endpoints which create new resources and have corresponding `GET` endpoint as speced in JSON API - https://jsonapi.org/format/#crud-creating-responses-201. Specifically:
/posts/
/pages/
/integrations/
/tags/
/members/
/labels/
/notifications/
/invites/
- Adding the header should allow for better resource discoverability and improved logging readability
- Added `url` property to the frame constructor. Data in `url` should give enough information to later build up the `Location` header URL for created resource.
- Added Location header to headers handler. The Location value is built up from a combination of request URL and the id that is present in the response for the resource. The header is automatically added to requests coming to `add` controller methods which return `id` property in the frame result
- Excluded Webhooks API as there is no "GET" endpoint available to fetch the resource
closes#12045
- When member's email is updated to an already existing email of different member it caused table's unique constraint error, which was not handled properly.
- Added handling for this error similar to one in members `add` method.
closes#11999
- When the routes.yaml file changes (manually or through API) we need
to store a checksum to be able to optimize routes reloads in the future
- Added mechanism to detect differences between stored and current routes.yaml hash value
- Added routes.yaml sync on server boot
- Added routes.yaml handling in controllers
- Added routes hash synchronization method in core settings. It lives in core settings
as it needs access to model layer. To avoid coupling with the frontend settings it accepts
a function which has to resolve to a routes hash
- Added note about settings validation side-effect. It mutates input!
- Added async check for currently loaded routes hash
- Extended frontend settings loader with async loader. The default behavior of the loader is
to load settings syncronously for reasons spelled in 0ac19dcf84
To avoid blocking the eventloop added async loading method
- Refactored frontend setting loader for reusability of settings file path
- Added integrity check test for routes.yaml file
no issue
- The async/await syntax makes it easier to reason about the code. Because adding 'Location' header is in the works it's a prep-work in a sense
no issue
- Updated magic link generation and validation methods for email update API to handle new support address
- Updated importer to ignore the new support address as it can only be updated via verification
- Updated members service to listen on settings edit for new support/reply address fields as well
- Updated tests to include the new settings
closes#12167
- Tags API v2 was ignoring `count.posts` include parameter.
- Regression was introduced with a3f693b472
- Introduced regression tests across all Content API versions to avoid similar bug in the future
no issue
- When an import was done and there were no "global labels" present Ghost created generic `import-[data]` label which later helped to find a specific batch of imported data
- It did not make sense to create such generic label when user provided their own unique label
- The rules that work now are:
1. When there is no global provided Ghost generates on and removes it in case there are no imported records
2. When there is a unique new global label provided no new label is generated, but the label stays even if there are no imported records
no issue
- When no members are succesfully imported through CSV import process the import label should not be created. Otherwise after multiple failed attempts to import there are orphaned labels in the system
* 3.30.2:
Bumped @tryghost/members-api to 0.28.1 in lockfile
Bumbed @tryghost/members-api to 0.28.1
🐛 Fixed unable to delete member when stripe is connected
refs #12127
- Adds new `editSubscription` endpoint for members admin API which allows updating individual subscription for a member - `PUT /members/:id/subscriptions/:subscription_id/`
- `editSubscription` has same permissions as member's `edit` endpoint
- Currently allows toggling of cancellation at period end for an active subscription
no issue
- When batch insert fails handling should be more granular and aim to retry and insert as many records from the batch as possible.
- Added retry logic for failed member's batch inserts. It's a sequential insert for each record in the batch. This implementation was chosen to keep it as simple as possible
- Added filtering of "toCreate" records when member fails to insert. We should not try inserting related members_labels/members_stripe_customers/members_stripe_customer_subscriptions records because they would definitely fail insertion without associated member record
no issue
- When stripe is disconnected and there are Stripe-connected records present in imported set they should not be processed and proper error should be thrown
no-issue
* Added bulkAdd method to Member,Customer&Subscription model
This allows us to keep the db access in the model layer
* Updated @tryghost/members-api to 0.27.2
This includes fixes for rate-limiting of requests, and exposes necessary
Stripe methods for creating customers and complimentary subscriptions,
without affecting the database.
* Refactored importer to parallelise tasks where possible
By parallelising our tasks we are able to improve the speed at which the
entire import completes.
no issue
- Additional validation is needed for imported data because in case of bulk insertions (through knex) we bypass model layer validation - this could lead to invalid data in the database, which would be hard to fix.
- Chose validation method we use for other endpoints - through JSON Schema. It proved to be very performant (200ms overhead for 50k records). When comparing it with iterative method (validating each record separately) this was adding about 17s of overhead.
- Refactored returned values from "sanitizeInput" method to encapsulate more logic so that the caller doesn't have to calculate amount of invalid records and deal with error types
- Whole sanitizeInput method could now be easily extracted into separate module (somewhere close to members importer)
- Bumped members-csv package. It is meant to handle empty string values - '' and null, which should allow validating member records more consistently!
no issue
- Updated sanitization logic to be self contained and return sanitieze input along with error stats
- This should give a nice place for validations to fit in
no-issue
* Added stripeSubscriptions relation to member model
This allows us to fetch the subscriptions for a member via standard
model usage, e.g. `withRelated: ['stripeSubscriptions']` rather than
offloading to loops and `decorateWithSubscriptions` functions, this is
more performant and less non-standard than the existing method.
* Updated serialize methods to match existing format
The current usage of `decorateWithSubscriptions` and the usage of
members throughout the codebase has a subscriptions array on a stripe
object on the member, this ensures that when we serialize members to
JSON that we are using the same format.
There is definitely room to change this in future, but this is an
attempt to create as few breaking changes as possible.
* Installed @tryghost/members-api@0.26.0
This includes the required API changes so that everywhere can use
members-api directly rather than models and/or helper methods
no issue
- The code in controller was becoming hard to reason about.
- Having a single module shows exactly how many dependencies are there to do an import for single batch.
- Having a separate module would make it easier to extract into it's own package in Members monorepo
no issue
- Member's labels have to have sort_order assigned when added/edited. This was lacking from batched importer.
- Implementation is based on logic used in model's base - e484709e73/core/server/models/base/index.js (L81-L86)
- deleted files under `core/server/lib/promise` and related test files
- added `@tryghost/promise` as a dependency
- fixed all local requires to point to the new package
no issue
- New Member API batched import is meant to be a substitution to current import
with improved performance while keeping same behaviore. Current
import processes 1 record at a time using internal API calls and times
out consistently when large number of members has to be imported (~10k
records without Stripe).
- New import's aim is to improve performance and process >50K
records without timing out both with and without Stripe connected
members
- Batched import can be conceptually devided into 3 stages which have
their own ways to improve performance:
1. labels - can be at current performance as number of
labels is usually small, but could also be improved through batching
2. member records + member<->labels relations - these could
be performed as batched inserts into the database
3. Stripe connections - most challanging bottleneck to solve because
API request are slow by it's nature and have to deal with rate limits of
Stripe's API itself
- It's a heavy WIP, with lots of known pitfalls which are marked with
TODOs. Will be solved iteratively through time untill the method can be
declared stable
- The new batched import method will be hidden behind 'enableDeveloperExperiments' flag to
allow early testing
refs 173e3292fa
- The bug was initially introduced in referenced commit. When request is done with `api_key` context, there should always be an `integration` object associated with it - 71c17539d8/core/server/services/permissions/parse-context.js (L36) . An `id` from `context.integration` not `context.api_key` has to be assigned to newly created webhook!
- The webhooks API is about to be declared stable in upcoming release, so no migration will be done
no issue
- When processing entries with new labels in parallel Bookshelf relations is trying to create them which caused unique key constraints to fail. To avoid the failure, all labels should be pre-created before proceeding with creating members
- Member limit code was duplicated in 2 places unnecessarily
- Also used member api code that fetched members and subscriptions fully hyrated when we only need a count
- Using a raw query significantly improves performance here
no-issue
This updates the Admin API Member resource to *not* cancel subscriptions
by default, and adds a `cancel` option. This can be used over HTTP by
including a `cancel=true` query parameter.
no issue
Having all members created during an import labelled with a specific "import label" is useful for later operations such as bulk delete/edit or simply recording how and when a member was created.
- automatically create a label with the date/time the members CSV import occurred and assign it to all imported members
- return the import label data in the API response so that clients can react accordingly such as automatically filtering the members list by the label once an import finishes
refs https://github.com/TryGhost/Ghost/issues/11971
- Added statusCode from bulk email provider to API response
- Updated error messages for different bulk email(mailgun) failure states
- Added `context` to preview mail API error message with mail provider's error message
closes#12033
- Added webhooks schemas and definitions.
- Added validation checking if integration_id is present when using session auth. This is needed to prevent orphan webhooks.
- Integrated webhook schemas into frame's validation layer.
- Added isLowerCase ajv keyword support. This is needed to be able to do isLowerCase validation using JSON Schema for webhooks.
closes#11994
- Adds support for ordering based on slug filter that contains a slug-is-in filter. It is applied only to Content API's resources - post, page, tag, author. The order is applied in the same order in which slugs appear in the filter.
- For, example providing following query parameter filter for any of the above resources: `?filter=slug:[kitchen-sink,bacon,chorizo]`, would filter them by these slugs and order in the same way defined in the filter
- Can be used in handlebars templates in following way: `{{#get "tags" filter="slug:[slugs,of,the,tags,in,order]"}}`
- The property conteining this new order is assigned to `autoOrder` instead of `rawOrder` intentionally. This explicit asstignment would allow distinguishing where the 'orderRaw' comes from the model or the API layer. Apart from adding necessary context this separation makes it easier to refactor separately model layer and API specific ordering in the future
- This commit also fixes default filtering for `author` resource in Content API. The serializer was never used before as it was missing from `serializers/index.js` module.
no issue
- Changes introduced to both API v3 and v2
- Makes sure to use the same integration_id as authenticated integration for the webhook's data.
- Makde it is impossible to create orphaned webhooks using token authentication
- Allowed only parent integration to edit it's children webhooks. Throwing permission error otherwise
no issue
- Adds new portal button settings to members site data for portal script
- Updates settings input/output serializers to handle portal icon image url
* tag '3.22.2':
v3.22.2
Updated Ghost-Admin to v3.22.2
Emitted all settings events on reinit of cache (#12012)
🐛 Updated access to be true by default in v3 API
Hardened members subscription migration against missing data (#12009)
closes#11990
- access should be a members feature, but it was already accidentally exposed to the theme layer
- it has now been added to the API even if members is disabled
- access defaults to true, unless members is enabled
- when members is enabled, access is set to the currently logged in members' access
refs 05f6faf846
- The logic in "Stripe" error detection depended on error's message string matching, which is not a good practice in general. Had fixed it do to exact match on "context" of the error we throw internally and left more extensive comment about why things are implemented the way they are.
no issue
Output serializer's url util was expecting `og_image` and `twitter_image` to be top-level attributes in the `attrs` object but they are actually nested under `posts_meta`.
- updated the code to use lodash's `get/set()` so that we can work with paths for easier handling of nested objects
- fixed unit tests where the mocked data under test did not match real-world data
closes#11574
- the current implementation of the access property has it frontend only, and wired up only in one place
- this leaves it only available in a handful of places, e.g. can't use it in a post loop or get helper
- the current implementation also fails logically if the html content of the post is blank
This fix moves the behaviour to the API
- this ensures the field is always available no matter what context you are in
- it also updates the logic to use the same membersHasAccess logic as is used to gate the post, so it's always correct
TODO: should reconsider the location of this code
refs https://github.com/TryGhost/Ghost/issues/10318
- Updates `boolean` serialization in v2/canary serializers to apply only for `boolean` type settings
- Updates `boolean` transformation in model layer `format`/`parse` to check on `boolean` type setting
- Removes error thrown on Read-only setting for settings edit endpoint
- Updates v2/canary input serializers to remove any Read-only settings (using RO flag) to avoid edits
- Added type/group mappings in the importer when pre-migration settings table import data is present
- Updates tests
* Updated members default settings
ref #10318
This pulls out the members_subscription_settings & stripe_connect_intgration settings into separate keys
* Updated usage of members_from_address
* Updated stripe_connect usage
* Updated members config to use new settings
* Updated members middleware to use isStripeConnected
* Updated members service to reload correctly
We reload the members-api instance when the related settings change, so
this makes sure we're listening to the correct settings changes
* Updated ghost_head helper to use new settings
* Updated theme middleware to use new settings
* Renamed members_allow_signup -> members_allow_free_signup
* Fixed tests after settings refactor
* Removed from direct key settings key
* Fixed regression tests for settings api
no issue
- Import error grouping wasn't working correctly when error didn't have defined context property
- The copy will be refined in final design review
refs TryGhost/Ghost#10318
refs 8fc526ff6
- This is symetric change to one done for v3 API (commited as 8fc526ff6)
- Added 'core' filtering for v2 API controller
refs https://github.com/TryGhost/Ghost/issues/10318
- `members_public_key` and `members_private_key` are now correctly grouped under `core` which is already filtered out so we don't need the extra filter to exclude them
refs https://github.com/TryGhost/Ghost/issues/10318
refs 1dc0405803
- Adds 1:1 mapping for filtering options to renamed settings "type" to "gorup"
- Ignores the name changes and any old types
- Detailsed type -> group mappings can be checked in the refereneced migration commit
refs https://github.com/TryGhost/Ghost/issues/10318
- There was a copy/paste error and we didn't have a test to pick it up. Will follow up with a regression test to make sure it doesn't happen again
refs #10318
refs 2614565d5a
- Renamed ghost_head/ghost_foot in settings to match the new names
introduced in migrations
- Above change lead to reshufling in the mappings in input/output
serializers
- Makes sure change is compatible with v2 API
refs https://github.com/TryGhost/Ghost/issues/10318
refs 2614565d5a
- Renames to match referenced migration renames
- Fixed API responses so they are consistent with newly renamed fields
- Not returning lang and timezone keys from settings in API v2 ther rest should be returned in API v3/canary
refs https://github.com/TryGhost/Ghost/issues/10318
- precursor to migrating from `settings.type` to `settings.group`
- renames `blog` type to `site`
- renames `bulk_email` type to `email`
- moves settings out of `site` (previously `blog`) into more appropriate groups such as `core` or individual feature groups
refs #10318
refs https://github.com/TryGhost/Ghost/pull/11942
- Removes force_i18n, permalinks, and members_session_secret usage from the codebase
- We deprecated these flasgs and have not used since Ghost v2. It's good time to remove them before we introduce bigger changes to how `settings` table opeartes.
- Fixed importer test. The test was meant to check if string values were converted properly, the check agains boolean didn't make much sense in this context, so removed it.
- Following this change are going to come ralated migrations to fix existing data (see ref)
no issue
- Moves out CSV parsing and serialization related code into separate package as a part of push to modularize Ghost repo.
- Next up is to remove `csv-parser` dependency from this new package
refs 7904c303a7
- Model's validation errors are sometimes returned as an array of errors, in those cases they should be flattened to before returning to the client. This way there is more insight into what went wrong during the import.
no-issue
pr: https://github.com/TryGhost/Ghost/pull/11930
- Upgraded @tryghost/members-api to 0.23.0
This version includes a new method hasActiveStripeSubscriptions
- Added /admin/members/hasActiveStripeSubscriptions
This can be used to determine whether or not we should allow removing
the stripe keys.
- Added /admin/settings/stripe/connect
This can be used to delete a Stripe Connect integration, provided
there are not active subscriptions
no issue
- `?formats` is useful when you want to get `html` back from the Admin API when creating/editing posts, otherwise you're forced to make a following `GET` request
no issue
- known failing case was Medium. Some *.medium.com articles triggered redirects but without cookies being stored/sent across the redirect requests it would cause an infinite redirect loop and we'd abort after 10 redirects
- use `got`'s [cookie support](https://github.com/sindresorhus/got/tree/v9.6.0#cookies) via `tough-cookie` so that we can create bookmark cards for medium.com and other sites with similar problems
no issue
- localhost check was causing an immediate "no provider" error when fetching oembed for a localhost url and bypassing the bookmark fallback even when the configured site lives on localhost
- allow `localhost:port` through in oembed endpoints when it matches the configured url
no issue
- there are various situations where we adapt/fix/improve our mobiledoc->html output over time but we didn't have a way of updating old content without manually editing the mobiledoc and saving, or running an expensive migration to re-render all old content
- this adds a `?force_rerender=true` query param to the `PUT .../admin/posts/:id/` endpoint that allows the `html` field to be re-generated without modifying the `mobiledoc` field contents
no issue
- NQL does not support the relationship setup that members->stripe customer<->stripe subscriptions uses so it wasn't possible to use the `filter` param to query against having an active subscription
- adds `customQuery` bookshelf plugin that allows customisation of SQL query used in `findPage` method by individual models
- use `customQuery` in Member model to set up joins and conditionals to select free/paid members when `options.paid` is present
- allow `?paid` param through API and permitted options for member model
no issue
- This new format allows to return additional metadata with failed import records. The data for invalid records is returned in following format:
```
{
count: {count_of_invalid_records},
errors: [{
message: "Members not imported. Members with duplicate Stripe customer ids are not allowed." // message field of the error
context: "Attempting to import members with duplicate Stripe customer ids." // context field of the error
help: "Remove duplicate Stripe customer ids from the import file, and re-run the import." // help field of the error
count: 2 // count of this specific error
}]
};
- Errors are grouped by their context fields because message fields sometimes can contain unique information like Stripe customer id, which would produce too many errors in case of bigger datasets.
no issue
- This endpoint is meant to be used for validation of imported members
- Main function at the moment is to validate if stripe_customer_id present in the dataset exists in connected Stripe account
closes#11008
- Updated @nexes/nql to 0.4.0
This version exports the mapKeyValues utility function
- Replaced nql-map-key-values with @nexes/nql util fn
Usage was found using `rg nql-map-key-values` and replaced globally.
- Deleted nql-map-key-values module in shared
Now that this module isn't referenced anywhere else, we can remove it,
relying solely on the util exported by @nexes/nql
no-issue
We've added a "mode" query param to the members_stripe_connect api auth
method, allowing the client to easily switch between live and test mode.
no issue
- When imported member contains stripe_customer_id data but there is no Stripe configured on the Ghost instance such import should faiil. The logic is consistent with one where import fails after not being able to find customer in linked Stripe account
- Fixed import stats to show import failures instead of "duplicate" when the validation error is of "Stripe" origin
no issue
- The aim was to extract format-csv to become an external dependency. After some analysis found out that native papaparse method `unparse` was achieving the same results with a lot better test coverage and stability. Because papaparse will become Ghost's default csv processor in near future decided to integrate papaparse instead of extracting module for the format-csv module's code, which would become redundant soon anyways.
- For reference papaparse will substitute current csv-parser lib because it's better performance and maturity.
- Performance comparison can be checked here - https://github.com/Keyang/csvbench#result . At the time of writing papaparse is rougly 40% faster than csv-parser
refs https://github.com/TryGhost/Ghost/issues/11414
Confirms if the fromAddress for sending member emails is valid and accessible using magic link flow, allowing owners to update full from address including domain change.
- Extends member service to handle magic link generation and validation for email update
- Updates existing setting endpoint to not directly update from address
- Adds new endpoint to send magic link to new address
- Adds new endpoint for validating the magic link when clicked and update the new email for from address
- Adds new email template for from address update email
refs https://github.com/TryGhost/Ghost/issues/11212
- if a bookmark card fetch is performed (either directly or from fallback) and the page does not have an extractable title, return a more specific error message than "No provider found for supplied URL."
no issue
- When the customer cannot be imported because they are missing from linked Stripe account or the linked account is incorrect one, these new messages should provide a better clue about what has caused the error and how to act on it.
no issue
- There is a need to be able to label certain import group of members
with custom labels. This will allow to distinguish/filter these newly
imported members.
- Allowed `POST /members/csv/` endpoint to accept `labels`
field parameter which assigns labels to every member from imported csv.
no issue
- Similarly to other additive api methods (e.g. members.add) returned more specific ValidationError with contex filled in with the reason why adding did not succed.
- This change is needed for more graceful label handling when adding new members through import
no issue
- Adding labels doesn't cause any content to invalidate, similarly to adding members. Unlike it's caunterpart - tags, there is no dependent "frontend" content that would become invalid
no-issue
This was because the API was recieving a default `null` value for the
stripe_connect_integration_token setting.
This also improves the logging for this error.
no issue
- added an `externalRequest` lib
- uses same underlying `got` module as our `request` lib
- uses `got`'s `beforeRequest` and `beforeRedirect` hooks to perform it's own dns resolution for each url that's encountered and aborts with an error if it resolves to a private IP address block
- includes a bypass for Ghost's configured url so that requests to it's own hostname+port are not blocked
- updated v2 and canary oembed controllers to use the `externalRequest` lib
no-issue
In order to issue a redirect we need access to the "raw" req/res
objects, which is why we must return the function which gets access to
them.
The members service is used to create the auth url and to update the
users session.
no-issue
Uses the members service to parse a stripe_connect_integration_token
setting and set the stripe_connect_integration based on that.
This change includes ignoring the stripe_connect_integration{,_token}
settings, as the token is never saved, and the integration can only be
set by using the token.
no-issue
There was some unused code here, the variable was never used, also we
were looping and collecting a list of errors, but only every using the
first one, so switched to the `find` method which stops iteration after
an element has matched.
- Represents that logging is shared across all parts of Ghost at present
* moved core/server/lib/common/logging to core/shared/logging
* updated logging path for generic imports
* updated migration and schema imports of logging
* updated tests and index logging import
* 🔥 removed logging from common module
* fixed tests
no issue
- adds `search` bookshelf plugin that calls out to an optional `searchQuery()` method on individual models to apply model-specific SQL conditions to queries
- updated the base model's `findPage()` method to use the search plugin within `findPage` calls
- added a `searchQuery` method to the `member` model that performs a basic `LIKE %query%` for both `name` and `email` columns
- allowed the `?search=` parameter to pass through in the `options` object for member browse requests
* moved `server/config` to `shared/config`
* updated config import paths in server to use shared
* updated config import paths in frontend to use shared
* updated config import paths in test to use shared
* updated config import paths in root to use shared
* trigger regression tests
* of course the rebase broke tests
no issue
- moves members stats generation for the admin graph from the client to the server
- outputs a basic totals count across a requested date range of 30, 90, 365 days, or all time. See below for the response shape
- leaves heavy lifting of the counts to the SQL engines - tested on a dataset of 100k members and query performance is <100ms
```
GET /ghost/api/canary/members/stats/?days=30
{
total: 100000,
total_in_range: 20000,
total_on_date: {
'2020-04-25': 19000,
'2020-04-26': 19500,
// continues until today's date
},
new_today: 200
}
```
* refactored `core/frontend/apps` to destructure common imports
* refactored `core/frontend/services/{apps, redirects, routing}` to destructure common imports
* refactored `core/frontend/services/settings` to destructure common imports
* refactored remaining `core/frontend/services` to destructure common imports
* refactored `core/server/adapters` to destructure common imports
* refactored `core/server/data/{db, exporter, schema, validation}` to destructure common imports
* refactored `core/server/data/importer` to destructure common imports
* refactored `core/server/models/{base, plugins, relations}` to destructure common imports
* refactored remaining `core/server/models` to destructure common imports
* refactored `core/server/api/canary/utils/serializers/output` to destructure common imports
* refactored remaining `core/server/api/canary/utils` to destructure common imports
* refactored remaining `core/server/api/canary` to destructure common imports
* refactored `core/server/api/shared` to destructure common imports
* refactored `core/server/api/v2/utils` to destructure common imports
* refactored remaining `core/server/api/v2` to destructure common imports
* refactored `core/frontend/meta` to destructure common imports
* fixed some tests referencing `common.errors` instead of `@tryghost/errors`
- Not all of them need to be updated; only updating the ones that are
causing failures
* fixed errors import being shadowed by local scope
no-issue
The flag currently defaults to `true` as we are still using stripe
direct. We expose it on the admin api config endpoint so that the
Ghost-Admin client can use it to conditionally render
refs https://github.com/TryGhost/Ghost/pull/11790
- reduced complexity by sticking to one email for both normal reset and forced reset (locked staff accounts)
- exposed `siteTitle` for use in any email templates
- updated email copy to be suitable for both types of password reset
- Fixed session invalidation for "locked" user
- Currently Ghost API was returning 404 for users having status set to "locked". This lead the user to be stuck in Ghost-Admin with "Rousource Not Found" error message.
- By returning 401 for non-"active" users it allows for the Ghost-Admin to redirect the user to "signin" screen where they would be instructed to reset their password
- Fixed error message returned by session API
- Instead of returning generic 'access' denied message when error happens during `User.check` we want to return more specific error thrown inside of the method, e.g.: 'accountLocked' or 'accountSuspended'
- Fixed messaging for 'accountLocked' i18n, which not corresponds to the
actual UI available to the end user
- Added automatic password reset email to locked users on sign-in
- uses alternative email for required password reset so it's clear that this is a security related reset and not a user-requested reset
- Backported the auto sending of required password reset email to v2 sign-in route
- used by 3rd party clients where the email is necessary for users to know why login is failing
Co-authored-by: Kevin Ansfield <kevin@lookingsideways.co.uk>
no issue
- Adds new endpoint on integration to refresh admin/content api key secret
- Allows owner/admin to refresh their content or admin API keys for an integration via Ghost Admin
- Adds a new `refreshed` event to actions table for anytime an api_key secret is refreshed
- All var declarations are now const or let as per ES6
- All comma-separated lists / chained declarations are now one declaration per line
- This is for clarity/readability but also made running the var-to-const/let switch smoother
- ESLint rules updated to match
How this was done:
- npm install -g jscodeshift
- git clone https://github.com/cpojer/js-codemod.git
- git clone git@github.com:TryGhost/Ghost.git shallow-ghost
- cd shallow-ghost
- jscodeshift -t ../js-codemod/transforms/unchain-variables.js . -v=2
- jscodeshift -t ../js-codemod/transforms/no-vars.js . -v=2
- yarn
- yarn test
- yarn lint / fix various lint errors (almost all indent) by opening files and saving in vscode
- grunt test-regression
- sorted!
no issue
- embed cards now store metadata including thumbnail urls in their payload
- we want to use this metadata to render video cards in emails
- by default oembed endpoints return fairly small thumbnail images that don't look great when blown up
- oembed supports a `maxwidth` query param that will instruct oembed providers to return larger sizes of the content if available
