Commit Graph

361 Commits

Author SHA1 Message Date
Fabien O'Carroll
b35f060bef Added settings for members 'free' price
refs https://github.com/TryGhost/Team/issues/637

The "free" price - when Members signup without using Stripe, should have
a name and description, so that it can be displayed in Portal in a
similar way to paid price's. As there is only ever one, and it is not a
fully fledged price, a setting makes more sense than a dedicated db
table.
2021-05-07 16:32:12 +01:00
Kevin Ansfield
322664a145
Added ability to send a newsletter to members with a certain label or product (#12932)
refs https://github.com/TryGhost/Team/issues/581
refs https://github.com/TryGhost/Team/issues/582

When publishing a post via the API it was possible to send it using `?email_recipient_filter=all/free/paid` which allowed you to send to members only based on their payment status which is quite limiting for some sites.

This PR updates the `?email_recipient_filter` query param to support Ghost's `?filter` param syntax which enables more specific recipient lists, eg:

`?email_recipient_filter=status:free` = free members only
`?email_recipient_filter=status:paid` = paid members only
`?email_recipient_filter=label:vip` = members that have the `vip` label attached
`?email_recipient_filter=status:paid,label:vip` = paid members and members that have the `vip` label attached

The older `free/paid` values are still supported by the API for backwards compatibility.

- updates `Post` and `Email` models to transform legacy `free` and `paid` values to their NQL equivalents on read/write
  - lets us not worry about supporting legacy values elsewhere in the code
  - cleanup migration to transform all rows slated for 5.0
- removes schema and API `isIn` validations for recipient filters so allow free-form filters
- updates posts API input serializers to transform `free` and `paid` values in the `?email_recipient_filter` param to their NQL equivalents for backwards compatibility
- updates Post API controllers `edit` methods to run a query using the supplied filter to verify that it's valid
- updates `mega` service to use the filter directly when selecting recipients
2021-05-07 11:56:41 +01:00
Kevin Ansfield
1ee97ccfbc
Fixed editors not being able to read labels when assigning member segments (#12934)
refs https://github.com/TryGhost/Team/issues/581

Editors are allowed to restrict post visibility and send emails to particular member segments, they need to be able to read labels so that they can select them in a member segment.
2021-05-07 10:51:44 +01:00
Kevin Ansfield
b08b95085e
Added default email recipients settings (#12925)
refs https://github.com/TryGhost/Team/issues/496

We want to give more control over the default selection of email recipients when publishing a post, to do that we need somewhere to store those settings. These settings are site-wide and intended for use by admins to control the default editor behaviour for all staff users. They _do not_ control API behaviour, if you want to send email when publishing via the API it's still necessary to explicitly opt in to that using the `?email_recipients_filter=` query param.

- new `editor` settings group to indicate that these settings only affect the UI rather than the API
- `editor_default_email_recipients` controls overall behaviour, string/enum with these allowed values:
  - `'disabled'`: no option to send email is shown in the editor's publishing dropdown
  - `'visibility'`: (default) selected member segment is dynamic and matches the post visibility filter
  - `'filter'`: specific member filter defined in `editor_default_email_recipients_filter` setting
- `editor_default_email_recipients_filter` is an NQL string for selecting members, used when `editor_default_email_recipients` is set to `'filter'`
  - default value is `'all'`
  - the segment string can be any valid NQL filter with the additional special-case values of `'all'` and `'none'`
2021-05-07 09:30:57 +01:00
Hannah Wolfe
273e220327 Moved i18n to shared
refs 829e8ed010

- i18n is used everywhere but only requires shared or external packages, therefore it's a good candidate for living in shared
- this reduces invalid requires across frontend and server, and lets us use it everywhere until we come up with a better option
2021-05-04 13:03:38 +01:00
Hannah Wolfe
829e8ed010 Expanded requires of lib/common i18n and events
- Having these as destructured from the same package is hindering refactoring now
- Events should really only ever be used server-side
- i18n should be a shared module for now so it can be used everywhere until we figure out something better
- Having them seperate also allows us to lint them properly
2021-05-03 17:14:52 +01:00
Hannah Wolfe
34d2cc1b0b Moved active theme to new theme engine service
refs: bf0823c9a2

- continuing the work of splitting up the theme service into logical components
- this is where it starts to get fiddly as the getActive function in themeService index is required across the frontend/backend mostly due to its use in the getApiVersion method
- for now left one usage of the getActive method in place in ghost-locals middleware ready for the next phase of the refactor, which will move some of the themeService index into a shared location
2021-04-23 15:28:50 +01:00
Daniel Lockyer
e9b21fdbd1 Updated bson-objectid calls to match API change
refs c873899e49

- as of `bson-objectid` v2.0.0, this library exports the function
  to generate an ObjectID directly, and then you need to use `.toHexString()`
  to get the 24 character hex string - 6696f27d82
- this commit removes all uses of `.generate()` and replaces with this
  change
2021-04-21 16:23:52 +01:00
Rishabh Garg
48a2d24497
Added stripe_price_id column to subscriptions table (#12881)
refs https://github.com/TryGhost/Team/issues/586

- Adds new `stripe_price_id` column to subscriptions table to store stripe price ids with `index`
- Populates `stripe_price_id` column value to current `plan_id` making the `plan_*` values redundant
- Updates tests
2021-04-20 16:37:59 +05:30
Kevin Ansfield
fff6a04c54
Migrated members_allow_free_signup setting to members_signup_access (#12886)
refs https://github.com/TryGhost/Team/issues/579

Currently the members signup setting is explicitly yes/no to allowing free members signup, with the implication that when set to "no" members is still active but members have to be created via Stripe or the admin API.

This change renames the setting and changes its type to allow more than a binary option.

- migration to create/update the new setting based on the old value
  - free signup = "all", no free signup = "invite"; matches the current UI for this setting
- rename setting everywhere it's used/tested against
- modify `getAllowSelfSignup()` used to configure members packages to only return `true` when the new setting is set to `'all'` to match behaviour to the older setting
- update importer to rename the setting when importing from an older Ghost version
2021-04-19 16:36:30 +01:00
Thibaut Patel
375c71fc6a 🔒 Added a way to hide the secret settings once they are set
issue https://github.com/TryGhost/Team/issues/621
2021-04-16 17:05:16 +02:00
Daniel Lockyer
d01d8988e2 Updated tests to reflect jwks-rsa v2 changes
refs https://github.com/auth0/node-jwks-rsa/blob/master/CHANGELOG.md#200---2021-03-01

- `jwks-rsa` switches from callbacks to async-await and so we need to
  change the tests to reflect this
- this commit moves from `getSigningKey` containing a callback to it
  being part of the async-await flow
2021-04-16 09:58:47 +01:00
naz
d964224312
Added settings keys to store OAuth related data (#12879)
refs https://github.com/TryGhost/Team/issues/618

- The `oauth_client_id` and `oauth_client_secret` are placeholders to store OAuths related data.
- The flag for `oauth_enabled` or anything along those lines was not added intentionally in favour of checking if the `oauth_client_id` & `oauth_client_secret` are null.
2021-04-16 10:02:21 +02:00
Fabien 'egg' O'Carroll
22924c62d4
Added permissions for the Products resource (#12875)
refs https://github.com/TryGhost/Team/issues/616

All roles which can publish posts should be able to read/browse products, as content gating
will be based on products going forward.

Creating, updating & destroying products will often make modifications to Stripe which requires
Administrator or Owner roles.

We also improve the permissions tests so that we no longer rely on things being in a particular
order.
2021-04-15 14:34:45 +01:00
Fabien 'egg' O'Carroll
15b7485a94
Added Product model and Member model relation (#12859)
refs https://github.com/TryGhost/Team/issues/586

- Member model now has `products` relation, sorted using `sort_order`, following convention from `labels`
- Product model has handling to set `slug` from name, following convention of Label model
- Updated filter plugin to handle filtering Member models by their `product` relations e.g. `product:[slug, slug]`
2021-04-08 18:01:49 +01:00
Rishabh Garg
94766c05bf
Added stripe_products and stripe_prices tables (#12858)
refs https://github.com/TryGhost/Team/issues/586

- Add the `stripe_products` table, so that we can map Stripe Products to Products in Ghost
- Add the `stripe_prices` table, so that we can associate Stripe Prices to Products table
2021-04-08 20:41:00 +05:30
Fabien 'egg' O'Carroll
25182b7b82
Added products and members_products tables (#12844)
refs https://github.com/TryGhost/Team/issues/586

- Add the products table, so that we can store Products in Ghost 
- Add the members_products table, so that we can associate Members w/ Products
- Use sort_order on the members_products table to follow the same convention in members_labels
- Populate the products table with a single product, using the name from the stripe_product_name setting
- Populate the members_products table with relations based on the status column of the members table

Populating the tables allows us to transition from the current system, which does not care about products, into the
new system, where Products are used to group members. The intention is that all existing paid members have the
same product
2021-04-08 14:15:30 +01:00
Naz
af6b73f085 Fixed typo in test case name 2021-03-30 17:54:53 +13:00
Naz
33a52d5850 Added test coverage for send_email in members API
no issue

- During an investigation of a possible Zapier bug discovered there was no coverage at all for sending emails for new member signups. This changeset adds coverage for basic case of `send_email` and `email_type` behavior
2021-03-30 17:54:53 +13:00
Rishabh Garg
72e8894eac
Cleaned old members stats endpoint (#12821)
no refs

- Removes old `/members/stats` endpoint in favor of new `/members/stats/count` in canary/v4 which captures members counts using new events table
- Removes tests for old `/members/stats` endpoint
- Added test for new `/members/stats/count` endpoint
2021-03-29 13:07:01 +05:30
naz
ffd866cedc
💥 Reduced default exports content for DB APIs (#12818)
refs https://github.com/TryGhost/Team/issues/555

- Export files included a lot of data which was not used in the importer, for example: members,  labels, migrations and many more. This lead to a lot of clutter in the import files and made it hard to reason about their purpose.
- The main purpose of exports - is to export importable resources. These are posts, tags, and users. The rest of data like members or migrations either have their own importer (like CSV importer for members) or does not and should not have any ways to be imported.
- These changes are in now way complete. It's a first step towards resource-based exports which could be properly versioned in the future on API level and not be a mirror of the DB structure.
- This is sort of a breaking change. But we are doing it because: (1) its an internal API that should not be used by external clients, (2) there was no public contract to have this API stable at any point, (3) we really need to get back the control over export files structure and size
- In case an external client was dependent on some structure of the exported json file they can still pass in ALL of previously exported data by passing table names in `include` query parameter.
2021-03-25 19:27:49 +13:00
naz
7f50083cc5
Fixed node version specific object key sorting (#12817)
refs 74cdbadedb

- Fixes failing test by enforcing ordering through `sort()` call. Doing this to avoid differences between node versions' `Object.keys()` ordering
2021-03-25 15:16:24 +13:00
Naz
74cdbadedb Fixed exporter tests using should.have.keys assertion
refs https://github.com/TryGhost/Team/issues/555
refs 080a8fc082

-  The `have.keys` assertion was not doing strict comparison neither provided any useful output when changed to `have.only.keys`.
- Rewrote the tests to use manual assertion through array comparison which checks exactly what it's supposed to and gives a visual diff in case there are any missing/extra properties in config
2021-03-25 13:22:34 +13:00
Naz
e290983042 Updated export fixture for Ghost 2.9
refs https://github.com/TryGhost/Team/issues/555

- This test fixure is up to date and contains real export from Ghost 2.9. This file should not be edited any more as it represents a "snapshot" of 2.9 (don't really know why this version was chosen to be tested, so didn't change the convention).
- Ideally in the future we should maintain snapshots of export from major releases (or at least first non-broken releases after major release) AND a snapshot of the latest ghost version export file (canary of a sort). This way we'll be able to build up confidence of older export files still working and keep good track in export file changes with every new version
- Updated db spec suite to remove "fixture" user before each import to reduce a variable amount of import errors (star from the same state). It has to be done through an API call because restarting Ghost instance to have clean slate before each test case times out the suite
2021-03-24 20:00:34 +13:00
Naz
4286fa8a53 Fixed skipped DB API regression tests
no issue

- The tests were skipped because schema changed and there was probably no time to figure out why they were failing.
- They work now!
2021-03-24 18:39:25 +13:00
Naz
0813bebac4 Updated export fixture for Ghost 4.0
refs https://github.com/TryGhost/Team/issues/555

- This test fixure is up to date and contains real export from Ghost 4.0. Previous file was just an old 2.x export file which was updated manually on "as needed bases"
- See 3240d4adf0 for more context
- Updated README.md for exports generations with more accurate instructions (previous one contained a typo)
2021-03-24 18:37:57 +13:00
Naz
32e1ea3679 Moved importer tests to regressions suite
refs 3240d4adf0

- Acceptance tests have low timeout limit and were not making it through with updated export files
2021-03-24 18:11:34 +13:00
Kevin Ansfield
929cb0afe9
🐛 Fixed plaintext and excerpt fallbacks having incorrect URLs after domain change (#12811)
closes https://github.com/TryGhost/Team/issues/571

- `post.plaintext` values were being transformed with the markdown URL transformer but that wasn't picking up the link format used in our plaintext fields resulting in absolute URLs being stored in the database rather than `__GHOST_URL__` URLs
  - meant that if the `url` config is changed then plaintext and other calculated fields that used it would have URLs that referred to the old domain rather than the new one
  - re-saving the posts would have updated it but that's not feasible to do manually for large sites
- bumped `@tryghost/url-utils` to a version that has plaintext transform utils and updated the post model's transform map
2021-03-23 16:56:14 +00:00
Kevin Ansfield
426cbeec0f
🐛 Fixed performance regression introduced in 4.1.0 (#12807)
closes https://github.com/TryGhost/Ghost/issues/12791
closes https://github.com/TryGhost/Team/issues/566

https://github.com/TryGhost/Ghost/pull/12787 introduced a significant performance regression due to a misunderstanding of when Bookshelf calls `.format()` ([related upstream issue](https://github.com/bookshelf/bookshelf/issues/668)). We expected `.format()` to only be called on save but it's also called when Bookshelf performs fetching and eager loading which happens frequently. `.format()` can be a heavy method as it needs to parse and serialize html and markdown so it should be performed as infrequently as possible.

- override `sync()` in the base model so we can call our own `.formatOnWrite()` method to transform attributes on `update` and `insert` operations
  - this was the only feasible location in Bookshelf I could find that is low enough level to not require modifying model instance attributes
  - gives models the option to perform heavy transform operations only when writing to the database compared to the usual `.format()` method that is also called on fetch in many situations
2021-03-23 09:11:24 +00:00
Naz
a71d5dc633 Renamed importer spec to match tested version
refs https://github.com/TryGhost/Team/issues/555

- Export/Import test suite clean up ctd. See previous commits for context
- Next up will introduce v3 and "latest" tests to be able to check how importer behaves when import files are used from a particular version
2021-03-23 00:42:22 +13:00
Naz
d30ecf70c0 Added latest export body generator
refs https://github.com/TryGhost/Team/issues/555

- Export/Import test suite clean up ctd. See previous commits for context
- Main goal here was to update latest JSON export data. Also hardened tests to make sure this fixture is updated whenever the endpoint changes structure
2021-03-23 00:42:22 +13:00
Naz
046c3445ce Renamed "lts" importer spec to "legacy"
refs https://github.com/TryGhost/Team/issues/555

- This name suits way better to describe what's actually being tested
2021-03-23 00:42:22 +13:00
Naz
efbaa4401e Renamed export body generator functions to match their versions
refs https://github.com/TryGhost/Team/issues/555

- This is a continuation of importer/exporter test suite cleanup. Check previous commits for more context
2021-03-23 00:42:22 +13:00
Naz
341b2bb604 Moved importer utils to general exporter utils
refs https://github.com/TryGhost/Team/issues/555

- Check previous commit for more context. This changeset is part of importer/exporter cleanup effort.
2021-03-23 00:42:22 +13:00
Naz
cc8fe3ad43 Removed importer suite assertion with no specific meaning
refs https://github.com/TryGhost/Team/issues/555

- Follow up to previous commit. Read it for full context.
2021-03-23 00:42:22 +13:00
Naz
653863bf0e Extracted v1 spec out of importer test suite
refs https://github.com/TryGhost/Team/issues/555

- Follow up to previous commit. Read it for full context.
- This step splits existing importer test suite into separate files based on current "describe sections". This is done to get the test suite size to somewhat comprehensive shape
2021-03-23 00:42:22 +13:00
Naz
0bc33d6816 Extracted LTS spec out of importer test suite
refs https://github.com/TryGhost/Team/issues/555

- Follow up to previous commit. Read it for full context.
- This step splits existing importer test suite into separate files based on current "describe sections". This is done to get the test suite size to somewhat comprehensive shape
2021-03-23 00:42:22 +13:00
Naz
4f83716cfe Extracted importer test suite utils
refs https://github.com/TryGhost/Team/issues/555

- As part of testing upcoming 4.0 release was looking into a robust way to test imports. The importer test suite is humongous, so started with small extractions.
- The goal is to have readable and easy to create version-specific "exports" that would allow for more convenient testing
2021-03-23 00:42:22 +13:00
Kevin Ansfield
afbe0c27fb
🐛 Fixed __GHOST_URL__ appearing in sitemaps (#12787)
closes https://github.com/TryGhost/Team/issues/552

Refactors URL transforms so they take place at the model layer rather than the API serializer layer. Continuation of the pattern created for the settings model in https://github.com/TryGhost/Ghost/pull/12738

- Added checks to all front-end tests to ensure output does not contain the magic replacement string
  - includes failing acceptance test for `__GHOST_URL__` appearing in sitemaps
- Removed all transform-ready URL transforms from API serializers
  - input serializers transform image urls relative->absolute to keep absolute-urls as the consistent "outside of the database" format
  - output serializers should not need to perform any URL transforms as that will be done at the model layer
- Added url transforms to models layer
  - removes knowledge from the API serializers which shouldn't need to know how data is stored internally in the database
  - makes absolute urls the consistent "outside of the database" URL format
  - adds transform step to the sitemap generator because the data used for that is fetched directly via knex which will not run through the bookshelf `parse()` methods
2021-03-18 17:16:37 +00:00
Thibaut Patel
ba511445df Fixed broken tests from previous commit
commit 63d55570a3
2021-03-17 16:18:48 +01:00
Naz
6b07d4b2a0 🐛 Fixed __GHOST_URL__ appearing in generated excerpts
refs https://github.com/TryGhost/Team/issues/467
refs a6f5eb71be

- When a generated excerpt is calculated for posts/page resources it uses raw model! to get the data. Model contains untranformed __GHOST_URL__ markup which has to be additionally processed before extracint an excerpt or use the transformed `plaintext` from available attributes (chose the latter to decrease complexity)
- Removed model dependency as `attrs` at this point of serialization should always contain the `plaintext` field. It's ugly and has an unsolved bug report here - https://github.com/TryGhost/Ghost/issues/10396. The reliance should be solved at some point, but definitely not a part of this issue
2021-03-17 19:12:40 +13:00
Kevin Ansfield
19d5448101
🐛 Fixed complimentary members' content gating (#12761)
no issue

Comped members were not able to view paid-member content because content gating was only looking for `member.status === 'paid'` which doesn't take into consideration members on a "complimentary" plan.

- added front-end acceptance tests for member access to posts
- updated content-gating check to take comped members into consideration
2021-03-15 19:13:48 +13:00
Hannah Wolfe
fdc043227a Updated fixtures for Ghost v4.0
- With 4.0 we have a brand new version of Casper, new fixtures and new default settings
- Fixture posts cover the key features and give users an introduction to how to use their site
- This all comes from the marketing and design teams to refresh the look and feel of Ghost and give users the best possible onboarding experience
Note: this fixture overhaul includes
- new content for new 4.0 features
- regenerated post content using our updated mobiledoc structure
- a switch from British to US English
2021-03-10 17:37:16 +00:00
Naz
a87d17e570 Updated utils for site endpoint response checks
refs 55dd16568b

- Fatfingered to add these files with a referenced commit
2021-03-09 22:12:05 +13:00
Naz
55dd16568b Added regression tests for Admin API /site endpoints
no issue

- We had a suspicion about a regression with these endpoints and there was no quick way to verify if these endpoints were failing due to a misconfiguration on the server or they broke generally for everyone
- Added tests as they were clearly lacking
2021-03-09 22:10:27 +13:00
Kevin Ansfield
021cfecb59
Fixed __GHOST_URL__ appearing in settings values after edit (#12738)
refs https://github.com/TryGhost/Ghost/pull/12736
refs https://github.com/TryGhost/Team/issues/467

knex's `parse()` method is only called on data when directly fetched from the db. This was causing problems when model instances are passed around via events for example because `.get('key')` will return data that was directly set on the model without having gone through the `parse()` transformations. The result of this inconsistency was settings appearing correct when Ghost started up but then being broken as soon as a setting was changed.

- moved absolute/relative->transform-ready URL transformations from the API input serializers to the model's `format()` method and replaced with a relative->absolute transform in API input serializers
    - results in consistency because `.get()` on a settings model will always return an URL
- removed transform-ready->absolute transforms from the API output serializers as that is now handled at the model-layer
2021-03-08 18:41:43 +00:00
Kevin Ansfield
a6f5eb71be
🏗 Changed internal URL storage format to use __GHOST_URL__ (#12731)
closes https://github.com/TryGhost/Team/issues/467

- switches to storing "transform-ready" URLs in the database
- transform-ready URLs contain a `__GHOST_URL__` placeholder that corresponds to the configured url that gives a few benefits
  - much faster and less memory intensive output transformations through not needing to parse html or markdown - the transform can be achieved using a straightforward regex find+replace
  - ability to change to/from or rename subdirectory without any manual updates to the database
- modified existing 4.0 url-transformation migration rather than adding another one and repeating the transformation on posts rows
2021-03-05 13:54:01 +00:00
Naz
f57115d3cc Fixed fixture creation order in webhooks tests
- The tests need to have 'integrations' as a first fixture created so that webhooks can relate to an integration
2021-03-04 10:56:58 +01:00
Naz
9995ae3616 Added support for locale key in Admin Settings API
refs https://github.com/TryGhost/Team/issues/509

- Allows to update and read 'locale' key along with the deprecated 'lang'
- In Ghost v5 the 'lang' key will be dropped and the migration in settings table will clean up the key name to match the one exposed through the APIs
2021-03-04 21:20:46 +13:00
Naz
e7e5f0ad15 Updated API versions used in regression test for webhooks
refs https://github.com/TryGhost/Team/issues/513

- Tests confused versions after doing a "tripplication"
- Extracted a global "API_VERSION" variable for each suite to make api version used in the test explicit
2021-03-04 13:06:16 +13:00
Naz
e63e4bb193 Hardened frontend tests checking API engine
refs 0f59537b96
refs https://github.com/TryGhost/Team/issues/221
refs 303046bc0a

- When the referenced changes were introduced they did not take into account upcoming engine versions and provided little guidance about what other areas might need to be checked and changed
- The last referenced commit shows an approximate scale of changes that might be beened when frontend engine defaults are modified in the future
2021-02-24 18:55:12 +13:00
Naz
11b2876c33 Removed unnecessary describe nesting
refs e71bb91541

- Simplified the file structure/nesting of the test suites
2021-02-24 14:32:03 +13:00
Naz
9dc71e1e84 Renamed site suite files to snake case patter
- The snake case pattern is used in all the rest of test suite file names
- Cleaned up these two files as they were sticking out in the folder I was touching
2021-02-24 14:22:22 +13:00
Naz
e71bb91541 Extracted api engine specific suites from site spec
refs https://github.com/TryGhost/Ghost/commit/abeabb764

- Continued work started in referenced commit in making sites_spec more manageable.
- Extracted out separate v2/v3/canary specs for each site configuration
- Now these files are easily diff'able and introducing new API version would include copying a file over instead of adding up to a gigantic file
2021-02-24 14:19:22 +13:00
Naz
abeabb7648 Moved site spec under separate folder
refs https://github.com/TryGhost/Ghost/commit/2f20930ff

- The aim is to split site test suite into separate files v2/v3/v4 after this move
- It is hard to grep or diff the file of this size.
- Having separate files per ghost-api engine will allow tracking changes between different API versions in the future
- It's a continuation of work started in refed commit
2021-02-24 14:02:13 +13:00
Naz
2f20930fff Moved vhosts tests out of site test suite
no issue

- The site_spec is a catch all suite with too many moving parts. While adding a new tests suite for v4 frontend configuration it felt overwhelming to add hundreds of lines to a >5000 line monstrocity
2021-02-24 13:55:54 +13:00
Hannah Wolfe
70ed998838 Replaced db.ready event with direct call to urlservice
refs dd715b33dc

- this is the last event that is used to trigger part of the standard boot process
- events make the code harder to read/reason about
- the urlservice is one of the most core and critical components in Ghost, possibly the biggest consumer of time and memory
- we want to have the work it is doing front and center so that we can improve it
2021-02-23 10:05:59 +00:00
Hannah Wolfe
f1be3418d9 Switched tests to use new boot method
- We currently have 2 boot systems in the codebase, until now the tests were still running against the old one
- Regression tests using the forceStart + subdir flag no longer work. These need reworking, but this _should_ be easier later
- Note those tests work fine if they were the first tests run, it is only the case of trying to restart the server with a subdirectory after starting without that doesn't work
2021-02-18 20:24:49 +00:00
Kevin Ansfield
29f34468f5
Updated mobiledoc tests for lazy loaded images (#12672)
refs 074ce4efb0
2021-02-18 19:49:40 +00:00
Kevin Ansfield
c7a29b4e7c
🐛 Fixed unexpected members-only content appearing in excerpt field (#12670)
closes https://github.com/TryGhost/Team/issues/468

- updated post-gating
  - clears excerpt if there's no access
  - rebuilds excerpt from free preview if paywall card is used and there's no custom excerpt
2021-02-18 14:12:06 +00:00
Kevin Ansfield
2c96df42ac
Added paywall card to mark end of free content preview (#12663)
closes https://github.com/TryGhost/Team/issues/466

- upgraded kg-default-cards to include paywall card
- extracted `htmlToPlaintext` from post model to shared util for re-use
- updated post-gating to set html+plaintext to the free preview if a paywall card has been used
  - re-generates plaintext from the truncated html using `htmlToPlaintext` util
- display free content in the `{{content}}` helper via the default CTA template
2021-02-17 23:00:26 +00:00
Fabien O'Carroll
7785fb6dc5 Refactored usage of pre v4 "unsplash" setting format
refs https://github.com/TryGhost/Ghost/issues/10318

- API changes introduced:

canary/v4 Admin API
GET /settings/ (browse)

+ "unsplash" present in response as boolean value

GET /settings/:settingName (read)

+ "unsplash" present in response as boolean value

PUT /settings/ (edit)

+ "unsplash" updates setting, accepts ONLY  boolean format

v3  Admin API
GET /settings/ (browse)

+ "unsplash" present in response with object value

GET /settings/:settingName (read)

+ "unsplash" present in response with object value

PUT /settings/ (edit)

+ "unsplash" updates setting, accepts either boolean or object formats

v2 Admin API
GET /settings/ (browse)

+ "unsplash" present in response with object value

GET /settings/:settingName (read)

+ "unsplash" present in response with object value

PUT /settings/ (edit)

+ "unsplash" updates setting, accepts object format
2021-02-18 10:55:59 +13:00
Hannah Wolfe
1b4ebce3b0 Split urlServiceUtils from testUtils & unified
- There is now just one way to wait until the UrlService is finished across all of our tests!
- db.ready is only called in the one utility + the actual UrlService tests
2021-02-17 17:36:27 +00:00
Naz
d0e0760dae Hardcoded labs to always return members:true
refs https://github.com/TryGhost/Ghost/issues/10318

- Because members is effectively "enabled" by default starting Ghost 4.0 have hardcoded labs setting to be such. The alternative of removing this key from labs would be equivalent to `labs.members === false` which is undesireable and would mean additional work on theme developer's side.
2021-02-17 19:55:16 +13:00
Naz
37ef40b46e Dropped support for labs setting parameter
refs https://github.com/TryGhost/Ghost/issues/10318

- `labs` setting is dropped from setting values as the use of JSON objec
to sore settings has been deprecated
- `labs` setting is no longer accepted as a paramter in the Settings API nor the
impoprter. The value is ignored if present in the POST/PUT requests and
returns 404 in case it is requested by key at `GET /settings/:key`
2021-02-17 19:55:16 +13:00
Kevin Ansfield
bb371017fe Updated expected mobiledoc handling missed importer tests
refs 2d091fa8f9
2021-02-16 12:34:27 +00:00
Kevin Ansfield
2d091fa8f9 Set mobiledoc.ghostVersion: '3.0' if missing on imported content
refs https://github.com/TryGhost/Ghost/issues/12646

- if the version is missing then the content was created before Ghost 4.0
- setting the version to `'3.0'` means it will continue to use the same rendering output so there are no unexpected breaking changes when migrating content
2021-02-16 12:08:40 +00:00
Kevin Ansfield
77f75808a7 Updated tests for new mobiledoc output
refs f6e7dac7bb
2021-02-16 11:42:07 +00:00
Kevin Ansfield
647529f56d Updated importer specs for mobiledoc.ghostVersion
refs ef76279084

- updated specs to expect `mobiledoc.ghostVersion` in blank documents
2021-02-15 19:32:36 +00:00
Kevin Ansfield
70486ccb4e
Updated importer to transform 'card-markdown' to 'markdown' (#12650)
refs https://github.com/TryGhost/Ghost/issues/12646

- `card-markdown` support is being removed from the default mobiledoc cards so we need to make sure imported content doesn't fail
- removed long-unused `cardName` payload property from tests
2021-02-15 16:42:01 +00:00
Fabien O'Carroll
8880cc82c7 Refactored usage of the pre v4 "slack" setting
refs https://github.com/TryGhost/Ghost/issues/10318

- Change/compatibility table for this changeset:

Importer

+ "slack_username"  import
- "slack_url" ignore
+ "slack.username" import
- "slack.url" ignore

v3 & canary/v4 Admin APIs
GET /settings/ (browse)

+ "slack_username" present in response
+ "slack_url" present in response
+ "slack" present in response

GET /settings/:settingName (read)

+ "slack_username" present in response
+ "slack_url" present in response
+ "slack" present in response

PUT /settings/ (edit)

+ "slack_username" updates setting
+ "slack_url" updates setting
+ "slack" !NOTE! updates setting unless "slack_username" or "slack_ur"l are set (updated keys take priority)

v2 Admin API
GET /settings/ (browse)

- "slack_username" NOT present in response
- "slack_url" NOT present in response
+ "slack" present in response

GET /settings/:settingName (read)

- "slack_username" ignored (404)
- "slack_url" ignored (404)
- "slack" present in response

PUT /settings/ (edit)

- "slack_username" ignored (no error)
- "slack_url" ignored (no error)
+ "slack" updates setting
2021-02-15 15:25:54 +13:00
Naz
e0178099a2 🏗 Altered slug unique constraint in posts table
refs https://github.com/TryGhost/Ghost/issues/12567

- Changing unique constraint from slug to slug+type should allow for posts and pages to be created with the same slug
- The constraint will be present on application layer for API v4 while we figure out how to deal with it in API v5
2021-02-09 13:31:12 +13:00
Fabien 'egg' O'Carroll
73f6fd8c51
Enabled Members for all sites (#12582)
no-issue

This removes all references to the members labs setting, any code that was run conditionally behind this flag now runs unconditionally. 

* Removed usage of Members labs flag
* Removed tests for Members disabled
* Added dynamic keypair generation for when setting is missing
2021-02-02 16:12:53 +00:00
Rishabh Garg
26ee648397
🏗 Flattened members subscriptions data in API (#12581)
no refs

- Updates member model serializer to directly set subscriptions on member object instead of `stripe.subscriptions`
- Updates all references to members subscriptions from nested `stripe.subscriptions` to `subscriptions`
- Updates v3 API serializer to still use `stripe.subscriptions`
- Updates tests
2021-02-02 16:12:53 +00:00
Fabien 'egg' O'Carroll
6af2706f10
Updated Admin API and Mega to use status flag (#12579)
no-issue

* Removed support for paid param from v3 & canary API
* Updated active subscription checks to use status flag
* Updated MEGA to use status filter over paid flag
* Removed support for paid option at model level
* Installed @tryghost/members-api@1.0.0-rc.0
* Updated members fixtures
2021-02-02 16:12:51 +00:00
Fabien 'egg' O'Carroll
a79ed1170d
Added status property to members (#12570)
refs #12160

This flag will allow us easier filtering of members via the API

* Added status column to members table

This flag will be used to determine if a member is free or paid, rather
than relying on joins with the customers and subscriptions tables.

* Added migration to populate members.status

As we add the column with a default value of "free" we only need to care
about the paid members here. We also preemptively handle migrations for
SQLite where there are > 998 paid members.
2021-02-02 16:11:34 +00:00
Thibaut Patel
b724b2be92
Removed the deprecated send_email_when_published field from the API
refs https://github.com/TryGhost/Team/issues/457
2021-02-02 16:11:34 +00:00
Thibaut Patel
746ab389ad
Removed the "type" filter from the GET settings api
refs https://github.com/TryGhost/Team/issues/454
2021-02-02 16:11:34 +00:00
Naz
e82a21cbcf Updated acceptance&regression test suite readmes
no reference

- The instructions in readmes were outdates. Updated with a little more accurate information (they still need a massive cleanup, but that should be a part of a separate effort)
2021-01-22 16:42:02 +13:00
Thibaut Patel
3127aac47c Added regression tests for the v3 endpoints
refs https://github.com/TryGhost/Team/issues/221
2021-01-22 16:32:03 +13:00
Fabien 'egg' O'Carroll
7fdddf34b3
🐛 Added multiple use grace period to tokens (#12519)
closes https://github.com/TryGhost/Ghost/issues/12347

This change allows a token to be used multiple times for the first 10
seconds after its initial use, this will stop dynamic link checking
software from invaliding magic links.
2021-01-18 17:03:41 +00:00
Rishabh Garg
592c1bd46c
Added FirstPromoter Integration (#12540)
no refs

Adds new FirstPromoter settings similar to amp, which allows sites to take advantage of FirstPromoter to launch their own member referral program natively.

- Adds new firstpromoter settings group
- Adds `firstpromoter` setting to group
- Adds `firstpromoter_id` setting to group for FirstPromoter referral tracking id
- Updated tests
2021-01-15 19:05:15 +05:30
Naz
2a6ae77221 Fixed linting error 2021-01-14 18:39:27 +13:00
Naz
1cec604318 Fixed update check tests
refs #12537

- Refactored overuse of rewire mocking blog-version.
- Fixed a bug introduced along the way when duplicate notifications errored instead of returning empty result
2021-01-14 18:30:09 +13:00
Kukhyeon Heo
7528ec8c3b
🐛 Fixed redirects "to" query params forwarding (#12333)
ref #10898

- The redirects configuration's `to` & `from` URL parameters used to ignore it's query string parameters, which resulted in unexpected behavior
- Current changeset only partially fixes the issue. Now `to` URL's query parameters always take precedence over incoming query parameters and the rest of query parameters are passed through.
2021-01-05 14:11:06 +13:00
Fabien O'Carroll
32fe260763 Updated canary members controller to use new Importer
no-issue

This completely replaces the old import functionality with the new
importer!
2020-12-09 19:25:57 +00:00
Kevin Ansfield
748a8e0d0f
Added email_open_rate order option to members API (#12462)
refs https://github.com/TryGhost/Ghost/issues/12421

- add `orderRawQuery` function to members model so that we can ensure members with an open rate are ordered before members without an open rate no matter the order direction chosen
- added `email_open_rate` to members in the test fixtures to allow testing of order
2020-12-08 12:43:48 +00:00
Kevin Ansfield
9ff7423b2b
Added email.open_rate order option to posts api (#12439)
refs https://github.com/TryGhost/Ghost/issues/12420

- updated `order` bookshelf plugin's `parseOrderOption()` method to return multiple order-related properties
  - `order` same as before, a key-value object of property-direction
  - `orderRaw` new property that is a raw SQL order string generated from `orderRawQuery()` method in models
  - `eagerLoad` new property that is an array of properties the `eagerLoad` plugin should use to join across
- updated `pagination.fetchAll()` to apply normal order + raw order if both are available and to handle eager loading / joins when `options.eagerLoad` is populated
- updated post model to include details for email relationship and to add `orderRawQuery()` that allows `email.open_rate` to be used as an order option
2020-12-03 20:13:37 +00:00
Daniel Lockyer
4ef019d88d Deleted v3 regression tests
- at the time of writing, the v3 API === canary API
- we have both v3 + canary regression tests, which are nearly the same
  but there are slight deviations that we keep missing when adding new
  tests
- the canary tests are actually describing functionality of the v3 API
- therefore, we should be ok to delete the v3 regression tests for now
- when v3 is stable, we can copy the canary tests back to v3
2020-12-02 18:27:29 +00:00
Daniel Lockyer
622a57fc26
Added missing canary test present in v3 tests
- this test was present in the v3 test suite but not in the canary ones
- given v3 == canary currently, these tests should essentially be the
  same
- copied the test over
2020-12-01 09:35:24 +00:00
Kevin Ansfield
5c5bf2d3b0 Fixed oembed tests being network dependent
no issue

- some tests started failing locally because example.com was not resolvable
- mocked dns lookup for all tests so that we're always testing against expected public/private IP address blocks
2020-11-30 21:44:15 +00:00
Kevin Ansfield
ea77f3f349
Added "email_track_opens" setting (#12404)
refs https://github.com/TryGhost/Ghost/issues/12390

- adds "email_track_opens" setting to the `email` group, defaulting to `true`
2020-11-23 18:36:12 +00:00
Fabien 'egg' O'Carroll
2600a20f54
Added new settings for members signup redirects (#12370)
refs #12366

Adds two new settings for redirecting members after signup, based on if
they're paid or free, defaulting to the current redirect of "/"
2020-11-17 17:17:54 +00:00
Fabien 'egg' O'Carroll
3f9053ae94
Added setting for newsletter footer content (#12359)
no-issue

We want to give users to ability to customise the content of their newsletter, and the first step
toward that is a setting in which we can store text or html to embed in the template
2020-11-13 13:18:47 +00:00
Fabien 'egg' O'Carroll
4604ba1587
Fixed backward compatibility for send_email_when_published (#12357)
no-issue

* Handled send_email_when_published in Posts API

This restores backwards compatibility of the Posts API allowing existing
clients to continue to use the `send_email_when_published` flag. This
change uses two edits, which is unfortunate. The reason being is that
this is an API compatibility issue, not a model issue, so we shouldn't
introduce code to the model layer to handle it. The visibility property
of the model is used to determine how to fall back, and because it can
be left out of the API request, and relies on a default in the settings,
we require that the model decide on the `visibility` before we run our
fallback logic (or we duplicate the `visibility` default at the cost of
maintenance in the future)

* Dropped send_email_when_published column from posts

Since this column is not used any more, we can drop it from the table.
We include an extra migration to repopulate the column in the event of
a rollback

* Updated importer to handle send_email_when_published

Because we currently export this value from Ghost, we should correctly
import it. This follows the same logic as the migrations for this value.

* Included send_email_when_published in API response

As our v3 API documentation includes `send_email_when_published` we must
retain backward compatibility by calculating the property.

* Fixed fields filter with send_email_when_published

* Added safety checks to frame properties

Some parts of the code pass a manually created "frame" which is missing
lots of properties, so we check for the existence of all of them before
using them.

* Fixed 3.1 migration to include columnDefinition

We require that migrations have all the information they need contained
within them as they run in an unknown state of the codebase, which could
be from the commit they are introduced, to any future commit. In this
case the column definition is removed from the schema in 3.38 and the
migration would fail when run in this version or later.
2020-11-11 13:03:41 +00:00
Peter Zimon
215bfd0a7a
Added newsletter design settings (#12352)
refs https://github.com/TryGhost/Ghost/issues/12355

- Adds new default settings for newsletter customisations - `newsletter_show_badge`, `newsletter_show_header` and `newsletter_body_font_category`
- Adds migrations to update group for new settings
- Add migration to update settings based on existing config value for newsletter settings
- Passes new newsletter settings to newsletter template and updates design based on them
- Fix tests
2020-11-11 18:26:11 +05:30
Fabien 'egg' O'Carroll
6140a98351
Updated newsletter functionality to use email_recipient_filter (#12343)
no-issue

* Used email_recipient_filter in MEGA

This officially decouples the newsletter recipients from the post
visibility allowing us to send emails to free members only

* Supported enum for send_email_when_published in model

This allows us to migrate from the previously used boolean to an enum
when we eventually rename the email_recipient_filter column to
send_email_when_published

* Updated the posts API to handle email_recipient_filter

We now no longer rely on the send_email_when_published property to send
newsletters, meaning we can remove the column and start cleaning up the
new columns name

* Handled draft status changes when emails not sent

We want to reset any concept of sending an email when a post is
transition to the draft status, if and only if, and email has not
already been sent. If an email has been sent, we should leave the email
related fields as they were.

* Removed send_email_when_published from add method

This is not supported at the model layer

* Removed email_recipient_filter from v2&Content API

This should not be exposed on previous api versions, or publicly

* Removed reference to send_email_when_published

This allows us to move completely to the email_recipient_filter
property, keeping the code clean and allowing us to delete the
send_email_when_published column in the database. We plan to then
migrate _back_ to the send_email_when_published name at both the
database and api level.
2020-11-06 17:32:23 +00:00
Kukhyeon Heo
5715aa2155
Added .yaml format support in redirects configuration (#12187)
closes #11085

- Ghost has been using YAML format for other configurations (e.g. routes). The plan is to move to this format for all user-edited settings files. By default JSON format is still used in Ghost Admin API v2/v3, but will be changed to YAML in API v4. Check referenced issue for more context.
- New format supports all the features available before. The main noticeable change is the structure of config file. It is now grouped by redirect HTTP code instead of specifying `"permanent": true | false` attribute for each config property. Example format for YAML config:
```
302:
  /from-url/: /to-url/

301:
  /category/([a-z0-9\-]+)/i: /tag/$1/
  /v([0-9\.]+)/docs/([a-z0-9\-]+)/i: /docs/$2/
```
- Added 2 new endpoints: `POST redirects/upload` and `GET redirects/download`. These serve as an alias to current GET/POST `/redirects/json. "upload/download" naming pattern is introduced to match the convention with other resources that can be uploaded and downloaded (images, themes etc.). `/redirects/json`  endpoints will be removed in Admin API v4
- The parsing code from `custom-redirects.js` has been moved to `frontend/services/redirects/settings.js`. This location is more appropriate for this logic and eventually `custom-redirects.js` middlewear might be moved into "frontend" as this middlewear plays a role mostly effecting that area.
2020-11-04 12:08:32 +13:00
Rishabh Garg
8ad11fe082
Enabled Portal (#12317)
no refs

[Portal](https://github.com/TryGhost/Portal) is a new drop-in script to make the bulk of Ghost membership features work on any theme out of the box, which was under a developer flag so far. This release removes the flag for Portal and makes it included as default for any members-enabled Ghost site. The Portal script is backward compatible with old public members script and existing Members-enabled themes should notice no change.

- Removes Portal config flag as Portal is now enabled by default
- Removes old members script as Portal is backward compatible with it
- Changes `{{content}}` helper to show default CTA in case of restricted content access
- `accent_color` setting is no more behind the dev experiment flag and included by default
- Adds migration to switch off Portal button setting for all existing sites which don't have Portal enabled in beta
2020-11-03 14:36:21 +05:30
naz
a2bfef53de
🐛 Fixed post resource filtering by posts_meta table fields (#12307)
refs #11572

- Filtering by fields coming from posts_meta table did not work for post resources. This was due to lack of support for these types of operations on NQL layer. The approach taken here is using same way filtering was done for many:many relations and generates a `WHERE IN` filtering clause. In the future we could look into adding preloading of 1:1 relations which should allow getting rid of `WHERE IN` in favor of `JOIN` and filtering directly by field names. 
- Changed structure of `EXPANSIONS` filter configuration. Current approach was based on "bag of all the things". Such structure will become problematic as more fields are added. For example, adding all the fields from 1:1 relation posts:posts_meta might collide with any other relations that would have similar naming like meta_description from tags table (if it were was added).
- Bumped nql version to 0.5.0. This adds filtering support to 1:1 relations
- Added filter expansions which can be unique per model Previous approach with single global expansions lookup wasn't working in case different models would need to declare expansion for same field names. Having a `filterExpansion` method per model works in a similar convention other filter related model methods do (e.g. enforcedFilters, defaultFilters)
2020-11-02 12:53:57 +13:00
Naz
635580f291 Fixed "no-shadow" linting in a test suite
refs 1439219
2020-10-20 13:19:33 +13:00
Naz
143921948d Fixed "no-shadow" eslint warning in tests
refs b6728ecb0f

- The "no-shadow" eslint rune was introduced into ghost's eslint plugin (referenced commmit), which resulted in flood of warning in console output when linting the project codebase.
- This cleanup is aiming to make any new linting issues more visible. Follow up commits will contain similar cleanups in other parts of the codebase
2020-10-19 17:45:26 +13:00
Kevin Ansfield
15afed4b81
Added snippets permissions migration and fixtures (#12283)
no issue

- all staff users can browse/read snippets so they're usable in the editor for everyone
- only administrators, editors, and admin integrations are able to create/edit/delete snippets
2020-10-16 12:53:09 +01:00
Naz
34875b85f3 Added test coverage for members search
no issue

- This test confirms there is no 404 returned when search fails to find any results. It's important to return a 200 in this case as some API clients (e.g. Zapier integration) could treat non-2xx responses as errors
2020-10-16 13:10:40 +13:00
naz
392140cb36
Allowed for comped field when creating a member through Members API (#12278)
closes #12273

- `comped` field has been allowed when editing a member or importing from a CSV. There has been a usecase (Zapier Integration) for API client to create a member with "Complimentary" plan, which made this change necessary
- Previously the logic for comped field was to skip and continue member record creation if Stripe was not connected. Now we throw an error - same as the one we have been throwing before when stripe_customer_id field was passed in. The implication of this change is that we won't be creating any record now if comped === true and Stripe is disabled. 
- Bumped admin-api-schema-package. Contains `comped` schema change so this field gets passed through to controller
2020-10-14 13:24:09 +13:00
Naz
39e403d176 🐛 Fixed meta attributes calculation on post preview
closes https://github.com/TryGhost/Ghost/issues/12247

- Internal preview controller was lacking "mapping" call to post object which handled not only missing meta attribute information but lots of other mappings (e.g. users, tags, etc.)
- Have added a regression test to catch issues like this in the future
2020-10-05 17:52:40 +13:00
Fabien 'egg' O'Carroll
854a41e556
🐛 Fixed CSV import json-schema email validation (#12239)
no-issue

By using the "email" validation, we were validating emails in CSV
imports using a different validator to the rest of the API. AJV's built
in email validation was failing on emails with "special" characters,
such as letters with an umlaut above them.

This commit brings the validation for CSV imports in line with the rest
of the API.
2020-09-29 12:51:35 +01:00
Naz
af516e130c Fixed error message checks in members regression test
refs 6f1abc610a

- Additional period `.` was introduced in referenced commit which broke these tests
- The period was added to follow general convention of ending error messages with a perio (in some situations validation message didn't make sense without proper punctuation)
2020-09-24 15:31:19 +12:00
naz
bbcc83dadb
Added support for ordering Post API resources by fields coming form posts_meta table (#12226)
refs #11729

- When ordering is done by fields from a relation (like post's `meta_title` that comes form `posts_meta` table), Bookshelf does not include those relations in the original query which caused errors. To support this usecase added a mechanism to detect fields from a relation and load those relations into query. 
- Extended ordering to include table name in ordered field name.  The information about the table name is needed to avoid using `tableName` within pagination plugin and gives path to having other than original table ordering fields (e.g. order by posts_meta table fields)
- Added test case to check ordering on posts_meta fields
- Added support for "eager loading" relations. Allows to extend query builder object with joins to related tables,
which could be used in ordering (possibly in filtering later). Bookshelf does not support ordering/filtering by proprieties coming from relations, that's why this kind of plugin and query expansion is needed
- Added note about lack of support for child relations with same property names.
2020-09-24 13:32:40 +12:00
Nazar Gargol
5766b39be6 Fixed failing regression tests
refs 32b37d7ba8

- Some of the regression tests were broken after changes in referenced commit
2020-09-22 16:13:57 +12:00
Nazar Gargol
32b37d7ba8 Improved error messaging for password reset process
refs #11878

- When password reset link is invalid previous messaging left the user
without clear information about why the reset failed and what they could do about it.
- Updated messaging around password reset tokens including detection of
when password token has invalid structure, has expired or has already
been used
2020-09-22 15:45:19 +12:00
Fabien 'egg' O'Carroll
812e4b682f
Added SingleUseToken model (#12215)
no-issue

This is a model for the tokens table, which handles the single use
aspect by customising the `findOne` method to automatically destroy the
model after reading from it
2020-09-18 15:05:56 +01:00
Nazar Gargol
d177852d1f Fixed posts_meta relation in test fixtures
refs d9dbcf6147

- Fixes broken commit
2020-09-17 20:46:45 +12:00
Nazar Gargol
e9204c9168 Updated invitations test coverage with more specific checks
refs https://github.com/TryGhost/Ghost/issues/11878

- Updated test cases in preparation to improve messaging in referenced
issue.
2020-09-16 14:24:24 +12:00
naz
cbdc91ce48
Added Location header to API's POST request responses (#12186)
refs #2635

- Adds 'Location' header to endpoints which create new resources and have corresponding `GET` endpoint as speced in JSON API - https://jsonapi.org/format/#crud-creating-responses-201. Specifically:
    /posts/
    /pages/
    /integrations/
    /tags/
    /members/
    /labels/
    /notifications/
    /invites/

- Adding the header should allow for better resource discoverability and improved logging readability
- Added `url` property to the frame constructor. Data in `url` should give enough information  to later build up the `Location` header URL for created resource.
- Added Location header to headers handler. The Location value is built up from a combination of request URL and the id that is present in the response for the resource. The header is automatically added to requests coming to `add` controller methods which return `id` property in the frame result
- Excluded Webhooks API  as there is no "GET" endpoint available to fetch the resource
2020-09-14 22:33:37 +12:00
Talha
afa976066a
🐛 Fixed table constraint error when updating member's email with an already existing email (#12178)
closes #12045

- When member's email is updated to an already existing email of different member it caused table's unique constraint error, which was not handled properly. 
- Added handling for this error similar to one in members `add` method.
2020-09-10 16:03:57 +12:00
Nazar Gargol
5582d030e3 Added routes.yaml content checksum storage to the db
closes #11999

- When the routes.yaml file changes (manually or through API) we need
to store a checksum to be able to optimize routes reloads in the future
- Added mechanism to detect differences between stored and current routes.yaml hash value
- Added routes.yaml sync on server boot
- Added routes.yaml handling in controllers
- Added routes hash synchronization method in core settings. It lives in core settings
as it needs access to model layer. To avoid coupling with the frontend settings it accepts
a function which has to resolve to a routes hash
- Added note about settings validation side-effect. It mutates input!
- Added async check for currently loaded routes hash
- Extended frontend settings loader with async loader. The default behavior of the loader is
to load settings syncronously for reasons spelled in 0ac19dcf84
To avoid blocking the eventloop added async loading method
- Refactored frontend setting loader for reusability of  settings file path
- Added integrity check test for routes.yaml file
2020-09-10 10:54:57 +12:00
Talha
c8a6939b2e
🐛 Fixed error caused by accepting invitation with existing email (#12172)
closes #12060

- A 500 error what happening when invited user provided an email that is associated with an existing user
- Additional validation for existing email address was added to prevent invalid data hitting db constraint error
2020-09-09 11:58:53 +12:00
naz
fcc6401445
Added routes_hash setting (#12171)
refs #11999

- The `routes_hash` setting will be used during the boot process to update the hash
of currently loaded routes.yaml file in case it's different from last restart
2020-09-07 21:54:55 +12:00
Kevin Ansfield
936d7a315a 🐛 Fixed incorrect status used for trial subscription query
refs 96c18edc82

- replaced `trialed` with `trialing` to correctly match the status used in Stripe
2020-09-02 13:07:48 +01:00
Kevin Ansfield
96c18edc82 🐛 Fixed members with Stripe trials not receiving paid-members newsletters
no issue

- members who have trial subscriptions added directly via Stripe will have a status of `"trialed"` in their Ghost subscription
- the `paid: true` filter was not taking that into account meaning trial users were not receiving newsletters sent to paid members even though they have a "paid" subscription
2020-09-02 00:34:26 +01:00
Rishabh Garg
bc2f6eb8e9 Added migrations for support and reply email address setting (#12163)
no issue

- Added default settings for the two new setting fields - `members_support_address` and `members_reply_address`
- Added migrations for setting group for new email settings
- Migration sets current from address as new support address default
- Added migration to set new support address same as from address
- Updated tests for new settings
- `members_support_address` - How members can reach for help with their account, public setting
- `members_reply_address` - Where you receive responses to newsletters
2020-08-31 17:16:53 +05:30
Nazar Gargol
0f81e5e84a Fixed regression tests for Tags APIs
refs #12167

- The reason for failing tests was  https://github.com/TryGhost/Ghost/issues/6104 . As we already test for ordering elsewhere, didn't include more "if/else" logic in tests just for the sake of passing. Refactored them to be order-independent instead
2020-08-31 21:09:12 +12:00
Rish
8cf9dc05e3 Revert "Added migrations for support and reply email address setting (#12163)"
This reverts commit 2216019a86.
2020-08-31 14:25:29 +05:30
Nazar Gargol
eb8c0fcff9 🐛 Fixed Tags API v2 to return posts count
closes #12167

- Tags API v2 was ignoring `count.posts` include parameter.
- Regression was introduced with a3f693b472
- Introduced regression tests across all Content API versions to avoid similar bug in the future
2020-08-31 18:46:35 +12:00
Rishabh Garg
2216019a86
Added migrations for support and reply email address setting (#12163)
no issue

- Added default settings for the two new setting fields - `members_support_address` and `members_reply_address`
- Added migrations for setting group for new email settings
- Migration sets current from address as new support address default
- Added migration to set new support address same as from address
- Updated tests for new settings
- `members_support_address` - How members can reach for help with their account
- `members_reply_address` - Where you receive responses to newsletters
2020-08-27 20:27:50 +05:30
Nazar Gargol
bcca59ffbe Fixed regression tests for members importer
refs 262d33b73c

- Added missing test changes from referenced commit
2020-08-27 19:28:18 +12:00
Kevin Ansfield
a995e9cc89 Allowed custom and search queries to be used with .findAll()
no issue

- extract filtering of an collection into a separate function
- use extracted function in `findAll()` so that it's query behaviour matches `findPage()`
2020-08-27 01:09:07 +01:00
Nazar Gargol
9d65de4b10 Removed created import labels when no records were imported
no issue

- When no members are succesfully imported through CSV import process the import label should not be created. Otherwise after multiple failed attempts to import there are orphaned labels in the system
2020-08-25 19:23:05 +12:00
naz
3a594ce22e
Improved error handling for batch inserted member records (#12146)
no issue

- When batch insert fails handling should be more granular and aim to retry and insert as many records from the batch as possible.  
- Added retry logic for failed member's batch inserts. It's a sequential insert for each record in the batch. This implementation was chosen to keep it as simple as possible
- Added filtering of "toCreate" records when member fails to insert. We should not try inserting related members_labels/members_stripe_customers/members_stripe_customer_subscriptions records because they would definitely fail insertion without associated member record
2020-08-20 17:41:47 +12:00
Nazar Gargol
6512b98049 Updated flaky member import tests
no issue

- Because there's concurrency involved during member import the order cannot be guaranteed.
2020-08-19 16:43:58 +12:00
Rishabh Garg
2bac2c67cc
Granted Admin users ability to impersonate member (#12132)
refs #12126

- Adds migration to add impersonation permission to administrators
- Adds default permission fixture to allow administrators to read member impersonation urls
- Allows administrators to create member impersonation magic links
2020-08-18 14:17:56 +05:30
naz
51c9a50c4f
Added validation layer to members import endpoint (#12131)
no issue

- Additional validation is needed for imported data because in case of bulk insertions (through knex) we bypass model layer validation - this could lead to invalid data in the database, which would be hard to fix.
- Chose validation method we use for other endpoints - through JSON Schema. It proved to be very performant (200ms overhead for 50k records). When comparing it with iterative method (validating each record separately) this was adding about 17s of overhead.
- Refactored returned values from "sanitizeInput" method to encapsulate more logic so that the caller doesn't have to calculate amount of invalid records and deal with error types
- Whole sanitizeInput method could now be easily extracted into separate module (somewhere close to members importer)
- Bumped members-csv package. It is meant to handle empty string values - '' and null, which should allow validating member records more consistently!
2020-08-17 23:28:57 +12:00
Nazar Gargol
ef41c57974 Added test coverage for invalid member import emails
no issue

- This test should serve as a control for new batched importer
2020-08-13 22:14:52 +12:00
Fabien 'egg' O'Carroll
1294e3f92c
Replaced all usage of member models with members-api (#12117)
no-issue

* Added stripeSubscriptions relation to member model

This allows us to fetch the subscriptions for a member via standard
model usage, e.g. `withRelated: ['stripeSubscriptions']` rather than
offloading to loops and `decorateWithSubscriptions` functions, this is
more performant and less non-standard than the existing method.

* Updated serialize methods to match existing format

The current usage of `decorateWithSubscriptions` and the usage of
members throughout the codebase has a subscriptions array on a stripe
object on the member, this ensures that when we serialize members to
JSON that we are using the same format.

There is definitely room to change this in future, but this is an
attempt to create as few breaking changes as possible.

* Installed @tryghost/members-api@0.26.0

This includes the required API changes so that everywhere can use
members-api directly rather than models and/or helper methods
2020-08-12 14:17:44 +01:00
Nazar Gargol
7d3f6e32ca Addes sort order support for imported members
no issue

- Member's labels have to have sort_order assigned when added/edited. This was lacking from batched importer.
- Implementation is based on logic used in model's base - e484709e73/core/server/models/base/index.js (L81-L86)
2020-08-12 16:34:15 +12:00
Daniel Lockyer
5b471e1bbe Extracted promise libs and history into @tryghost/promise
- deleted files under `core/server/lib/promise` and related test files
- added `@tryghost/promise` as a dependency
- fixed all local requires to point to the new package
2020-08-11 18:44:21 +01:00
Daniel Lockyer
c9a5b28669 Extracted core/server/lib/security to @tryghost/security package
- code and tests were extracted out to this package
- deletes these files
- replaces all local requires, and adds it as a dependency
2020-08-11 14:06:50 +01:00
Fabien 'egg' O'Carroll
d15446593a
Added support for ON DELETE CASCADE to the schema (#12105)
no-issue

We are in the process of creating migrations to add foreign key constraints
and cascading deletes to the members_stripe_* tables to make listing members
and deleting members faster. As well as the migrations we need to update the
database schema so that new installations have the correct indexes and constraints.

Co-authored-by: Kevin Ansfield <kevin@lookingsideways.co.uk>
2020-08-05 13:20:30 +02:00
Nazar Gargol
60ae9e82f9 Fixed integration_id assignment for webhook when creating through API key auth
refs 173e3292fa

- The bug was initially introduced in referenced commit. When request is done with `api_key` context, there should always be an `integration` object associated with it - 71c17539d8/core/server/services/permissions/parse-context.js (L36) . An `id` from `context.integration` not `context.api_key` has to be assigned to newly created webhook!
- The webhooks API is about to be declared stable in upcoming release, so no migration will be done
2020-08-04 16:43:24 +12:00
Nazar Gargol
1b449f4f53 🐛 Fixed 500 error in webhooks API when modifying non-existing webhooks
closes #12064

- Handled permission check bug by returning 404, same way it is returned in other permissions related places when handling non-existing resource. Example - 60907a7ae4/core/server/models/relations/authors.js (L355-L358)
2020-08-03 23:08:47 +12:00
Kevin Ansfield
0c5c7b32b8 Fixed members import regression tests
refs 14c6968388

- updates expected member labels length asserts to account for auto-generated import label being attached
2020-07-26 18:57:11 +01:00
Fabien O'Carroll
c46475f5be Supported optionally cancelling subscriptions on delete
no-issue

This updates the Admin API Member resource to *not* cancel subscriptions
by default, and adds a `cancel` option. This can be used over HTTP by
including a `cancel=true` query parameter.
2020-07-24 16:01:06 +02:00
Fabien 'egg' O'Carroll
5144a0e09c
Updated Member model to cascade on destroy (#12077)
no-issue

Up until now we have left orphaned rows in members_stripe_* tables when
a member is deleted, this updates the destroy method so that we cascade
and remove any MemberStripeCustomer and StripeCustomerSubscription
models related to the Member.

This also adds regression tests for the new functionality as well as to
confirm the existing functionality of cascading to the members_labels
join table

This adds the relations of Subscription->Customer & Customer->Member
2020-07-23 18:21:10 +02:00
Hannah Wolfe
995a4eb46e 🐛 Fixed edit link for staff members
closes #12069

- the url in admin changed from team to staff, and wasn't updated in the frontend
2020-07-21 12:28:50 +01:00
Fabien 'egg' O'Carroll
6232981be7
🐛 Fixed importing Stripe Plans with amount 0 (#12062)
closes #12049

Stripe plans used to default to 0, and our new validation of plan
amounts were causing issues when importing from an older version of
Ghost, this updates the validation to be skipped when importing.

- Added regression test for importing plans
2020-07-20 14:59:23 +02:00
Nazar Gargol
a520cdad0b Added JSON Schema validations to Webhooks Admin API v3
closes #12033

- Added webhooks schemas and definitions.
- Added validation checking if integration_id is present when using session auth. This is needed to prevent orphan webhooks.
- Integrated webhook schemas into frame's validation layer.
- Added isLowerCase ajv keyword support. This is needed to be able to do isLowerCase validation using JSON Schema for webhooks.
2020-07-17 17:37:14 +12:00
Fabien O'Carroll
48eca5f618 Added amp_gtag_id setting migration & default
refs #11980

This lays the ground for the rest of the work surrounding Google
Analytics in AMP.
2020-07-15 13:43:25 +02:00
Nazar Gargol
ced3e28d60 🐛 Fixed cache invalidation for published posts with no content
closes #12015
refs 95880dddeb

- The bug was caused by falsy plaintext field assignment to empty string `''` when the html content was `null`. Because of the `setEmptyValuesToNull` function (referenced commit), there is no sense to assign empty string value to plaintext property, because it would still end up being `null`
- The `''` -> `null` conversion was confusing the model layer to think that some fields were changed, where in reality none did. This in turn lead to a bug with falsy cache invalidation
2020-07-15 17:41:24 +12:00
Nazar Gargol
a75e1e27db 🐛 Fixed cache invalidation when editing post_meta fields
closes #12016

- The change detection didn't work when editing post_meta fileds because we only check current model's `_changed` fields when performing `wasChanged()` check
- A solution was adding change tracking of post_meta relation to currently edited post model and overloading `wasChanged` method to check these fields as well
2020-07-14 23:48:36 +12:00
Fabien 'egg' O'Carroll
20a95de8f7
Added new stripe webhook settings (#12028)
no-issue

They will be used to store webhook information so that we can persist it between
boots and simplify the creation process of webhooks in members
2020-07-10 16:26:06 +02:00
Rish
d6f4f01eff Added new settings integrity tests
refs https://github.com/TryGhost/Ghost/issues/12026

- We made changes to settings table structure in 3.22 which added new columns for `group` and `flags`
- Any new setting needs explicit group and flag migrations since or they fallback to default group value of `core`
- This tests ensures
  - hash is updated in DB integrity test anytime default-settings are changed
  - that a migration is present by maintaining a whitelist of allowed core settings and failing if new setting is added without correct group migration
2020-07-10 18:32:54 +05:30
Nazar Gargol
d6267340a1 Updated Content API resource ordering to be same as slugs in filter
closes #11994

- Adds support for ordering based on slug filter  that contains a slug-is-in filter. It is applied only to Content API's resources - post, page, tag, author. The order is applied in the same order in which slugs appear in the filter.
- For, example providing following  query parameter filter for any of the above resources: `?filter=slug:[kitchen-sink,bacon,chorizo]`, would filter them by these slugs and order in the same way defined in the filter
- Can be used in handlebars templates in following way: `{{#get "tags" filter="slug:[slugs,of,the,tags,in,order]"}}`
- The property conteining this new order is assigned to `autoOrder` instead of `rawOrder` intentionally. This explicit asstignment would allow distinguishing where the 'orderRaw' comes from the model or the API layer. Apart from  adding necessary context this separation makes it easier to refactor separately model layer and API specific ordering in the future
- This commit also fixes default filtering for `author` resource in Content API. The serializer was never used before as it was missing from `serializers/index.js` module.
2020-07-10 18:33:00 +12:00
Nazar Gargol
173e3292fa Fixed integration_id handling in Webhooks Admin API
no issue

- Changes introduced to both API v3 and v2
- Makes sure to use the same integration_id as authenticated integration for the webhook's data.
-  Makde it is impossible to create orphaned webhooks using token authentication
- Allowed only parent integration to edit it's children webhooks. Throwing permission error otherwise
2020-07-08 16:54:31 +12:00
Rish
c0e3c8bcfe Added new settings for portal button customization
no issue

- Adds new portal settings - `portal_button_style`, `portal_button_icon` and `portal_button_signup_text`
- New settings allows customization of portal button
- Updates tests to include new settings
2020-07-07 14:06:04 +05:30
Daniel Lockyer
8956a9df35 Merged 3.22.2 into master
* tag '3.22.2':
  v3.22.2
  Updated Ghost-Admin to v3.22.2
  Emitted all settings events on reinit of cache (#12012)
  🐛 Updated access to be true by default in v3 API
  Hardened members subscription migration against missing data (#12009)
2020-07-06 16:19:55 +01:00
Hannah Wolfe
289c1b3e8a 🐛 Updated access to be true by default in v3 API
closes #11990

- access should be a members feature, but it was already accidentally exposed to the theme layer
- it has now been added to the API even if members is disabled
- access defaults to true, unless members is enabled
- when members is enabled, access is set to the currently logged in members' access
2020-07-06 16:04:29 +01:00
Daniel Lockyer
f65c425786 🐛 Fixed searching for free/paid members
no issue

- when searching for paid/free members, the `members_stripe_customers`
  table would be joined into the query on `members`
- this table also has a `name` and `email` field, so both MySQL and
  SQLite would complain about ambiguous fields in the query
- the result of this would be a 500 error thrown inside Ghost, and no
  useful response to the user
- this commit explicitly chooses the `members` table to check against,
  and also adds a test for this
2020-07-06 09:09:08 +01:00
Fabien 'egg' O'Carroll
37d577fa90
Remove references to bulk email settings (#12000)
refs #10318

* Ignored bulk_email_settings when importing
* Removed custom handling for bulk_email_settings
* Removed bulk_email_settings from default-settings
* Ignored bulk_email_settings when editing settings
2020-07-03 16:30:54 +02:00
Fabien 'egg' O'Carroll
9df217b04e
Added mailgun_{domain,api_key,base_url} settings (#11992)
refs #10318

 - Adds new default settings
 - Adds migration for populating settings with bulk_email_settings data
 - Fixes regression tests
2020-07-03 11:00:20 +02:00
Nazar Gargol
6902f86b40 Added field mapping support for members import CSVs
no issue

- This is a precursor work allowing client to send csv to JSON
field mapping hash as a `mapping` field parameter in the incoming
request.
2020-07-02 15:18:39 +12:00
Jonas Fabisiak
58ac033b97
Updated head to only output custom favicons (#11949)
closes #11932

- as per the issue, there is no need to output a <link> tag if the favicon is the default /favicon.ico, as all browsers automatically check for this 
- instead the favicon <link> is only output if a custom favicon has been set, telling the browser to look somewhere different to its default location
- some of the tests expected 3 links in the html head, but through the favicon change there are often only 2.

Co-authored-by: RenCloud <rencloud@pop-os.localdomain>
2020-06-30 15:07:08 +01:00
Hannah Wolfe
fa91c6c954
🐛 Moved access to an API property (#11967)
closes #11574

- the current implementation of the access property has it frontend only, and wired up only in one place
- this leaves it only available in a handful of places, e.g. can't use it in a post loop or get helper
- the current implementation also fails logically if the html content of the post is blank

This fix moves the behaviour to the API

- this ensures the field is always available no matter what context you are in
- it also updates the logic to use the same membersHasAccess logic as is used to gate the post, so it's always correct

TODO: should reconsider the location of this code
2020-06-30 14:46:46 +01:00
Fabien 'egg' O'Carroll
ee786aaa5d
Cleaned up members & stripe settings (#11957)
* Updated members default settings

ref #10318

This pulls out the members_subscription_settings & stripe_connect_intgration settings into separate keys

* Updated usage of members_from_address

* Updated stripe_connect usage

* Updated members config to use new settings

* Updated members middleware to use isStripeConnected

* Updated members service to reload correctly

We reload the members-api instance when the related settings change, so
this makes sure we're listening to the correct settings changes

* Updated ghost_head helper to use new settings

* Updated theme middleware to use new settings

* Renamed members_allow_signup -> members_allow_free_signup

* Fixed tests after settings refactor

* Removed  from direct key settings key

* Fixed regression tests for settings api
2020-06-29 16:22:42 +02:00
Kevin Ansfield
f45198245f Updated Settings.populateDefaults() to account for available columns
refs https://github.com/TryGhost/Ghost/issues/10318

`Settings.populateDefaults()` is run before migrations during Ghost's startup. This can cause problems when new settings table columns are added (and populated in `default-settings.json`) because `populateDefaults()` was using the model layer which assumes that those columns are available, resulting in `ER_BAD_FIELD_ERROR: Unknown column` type errors.

- query the database for the available `settings` table columns
- switch to using raw knex queries without Bookshelf for insertions so that we're in control of the columns that are added
- use `_.pick` to skip any properties in `default-settings.json` that do not match to an available column - those columns will be added and populated by later migrations
- moving away from using the model to insert settings has the side-effect of not emitting `settings.added/edited` and `settings.x.added/edited` events, this should be fine because `populateDefaults()` is called before anything else is set up and listening
- added a call to `populateDefaults()` in our knex-migrator "before migration" hook so that we have consistent db state across both startup initialised migrations and manually triggered knex migrations
2020-06-29 13:22:10 +00:00
Nazar Gargol
91e3630f36 Updated settings API v3 tests to check for correct types returned for specific keys
refs https://github.com/TryGhost/Ghost/issues/10318
refs 476fca6e5b

- Symetric change to one done in referenced commit which maps fields for API v2
2020-06-25 17:36:10 +12:00
Nazar Gargol
476fca6e5b Updated settings API v2 tests to also check for correct types
refs https://github.com/TryGhost/Ghost/issues/10318

- Added additional check for correct "type" fields in settings responses
- The list is based on pre-migration response from API v2, so makes it very clear there is no breaking change after the migration
2020-06-25 17:13:24 +12:00
Nazar Gargol
e394b5ad9c Added naive settings type options parameter support to settings API v2
refs TryGhost/Ghost#10318
refs 8fc526ff6

- This is symetric change to one done for v3 API (commited as 8fc526ff6)
- Added 'core' filtering for v2 API controller
2020-06-25 16:32:16 +12:00
Nazar Gargol
8fc526ff6e Added back compatible support for renamed settings types
refs https://github.com/TryGhost/Ghost/issues/10318
refs 1dc0405803

- Adds 1:1 mapping for filtering options  to renamed settings "type" to "gorup"
- Ignores the name changes and any old types
- Detailsed type -> group mappings can be checked in the refereneced migration commit
2020-06-25 00:55:40 +12:00
Kevin Ansfield
f08b5425e0 Fixed regression tests
no issue

- adds group and flags to the expected properties list for settings API responses
2020-06-24 12:14:06 +01:00
Kevin Ansfield
4a9e57c170
Added migration to add settings.{group,flags} columns (#11951)
refs https://github.com/TryGhost/Ghost/issues/10318

- `group`
  - to replace the `type` column, provides a more descriptive name for the columns use
  - for existing sites it will be populated by migrating data from the `type` column in a later migration
  - for new sites a minimal update has been added to `parseDefaultSettings()` to populate the `group` field when settings are created during startup - fixes the NOT NULL constraint on `settings.group`
- `flags`
  - signifies special handling that is different to other settings in a group
  - eg, `PUBLIC,RO` would indicate that the setting is available via unauthenticated endpoints and is read-only
2020-06-24 11:58:15 +01:00
Nazar Gargol
8f1905990d Added regression test coverage for editing deprecated settings
refs #10318
refs 118b7033e0

- Adds test coverage for fix introduced in referenedc commit
2020-06-24 22:40:57 +12:00
Nazar Gargol
f415389829 Updated admin admin v3 API test suite to match symmetric one in canary
no issue

- There are no functional differenec between canary and v3 APIs at the moment, thus these test suites should be kept in sync
2020-06-24 20:04:34 +12:00
Nazar Gargol
a21941ba51 Added settings ?type parameter to API regression tests
refs https://github.com/TryGhost/Ghost/issues/10318

- At the moment there were no test checking correct behavior of `?type` parameter. Given ongoing settings refactoring work expanded tests with some essential coverage
- Moved one of the test cases to be consistent with order in other suites
2020-06-24 20:01:28 +12:00
Nazar Gargol
b00369ac67 Fixed settings tests
refs 8a817050c5

- This change reflects field reshuffle done in referenced commit
-  NOTE: removed `members_email_auth_secret` key from respnse body validation as it seems like it should never have been exposed in the first place! This needs verification as it's a breaking change theretically
2020-06-24 14:55:15 +12:00
Nazar Gargol
8ea245fc7c Fixed importer mapping for renamed default_locale, active_timezone, ghost_* settings
refs #10318
refs 2614565d5a

- Adds importer mapping for fields as in referenced migration
- The intention is to allow exports from v2/v3 to still be compatilbe with current version
- Adds a mapper for deprecated fields and imports them with new settings "keys"
2020-06-24 14:19:52 +12:00
Nazar Gargol
f0a811e9fe Renamed ghost_head/ghost_foot to codeinjection_head/codeinjection_foot
refs #10318
refs 2614565d5a

- Renamed ghost_head/ghost_foot in settings to match the new names
introduced in migrations
- Above change lead to reshufling in the mappings in input/output
serializers
- Makes sure change is compatible with v2 API
2020-06-24 14:18:13 +12:00
Nazar Gargol
5f1060a8bf Renamed settings keys active_timezone to timezone
refs https://github.com/TryGhost/Ghost/issues/10318
refs 2614565d5a

- Renames to match referenced migration renames
- Fixed API responses so they are consistent with newly renamed fields
- Not returning lang and timezone keys from settings in API v2 ther rest should be returned in API v3/canary
2020-06-24 14:11:20 +12:00
Nazar Gargol
b78dbfbbc2 Removed use of deprecated setting flags from the codebase
refs #10318
refs https://github.com/TryGhost/Ghost/pull/11942

- Removes force_i18n, permalinks, and members_session_secret usage from the codebase
-  We deprecated these flasgs and have not used since Ghost v2. It's good time to remove them before we introduce bigger changes to how `settings` table opeartes.
- Fixed importer test. The test was meant to check if string values were converted properly, the check agains boolean didn't make much sense in this context, so removed it.
- Following this change are going to come ralated migrations to fix existing data (see ref)
2020-06-23 11:58:19 +12:00
Fabien O'Carroll
8a74cd9e11 Fixed Redirects API regression tests
no-issue

These tests were not testing the functionality correctly, but trying to
test it based on metadata, which can become out of date. This tests the
expected responses and is resilient for future changes to fixtures.
2020-06-16 10:24:13 +02:00
Fabien O'Carroll
c5b8dab523 🐛 Fixed custom redirects for subdirectory setups
no-issue

This issue only occurs when using custom redirects with a subdirectory
setup, and the path to be redirected from is expressed as a regex, and
the url that is being redirected to is not an external url.

The issue has a few components:

- Redirect paths as a regex generally use the ^ to ensure that they
  match the beginning of the path.

- The path that the regex is matched against conditionally excludes the
  subdirectory, specifically, the subdirectory is excluded for external
  urls

These combined means you end up with a regex like /^\/custom-redirect/
and a path like /subdir/custom-redirect, these will not match/replace
correctly, and you'll end in an infinite redirect loop.

The fix here is to *always* remove the subdirectory when testing regex's
and then conditionally adding it back *only* for the redirect, and only
if it is an internal redirect
2020-06-16 10:24:13 +02:00
Nazar Gargol
5f00619d1a Renamed members import/export endpoints
refs cb26fd9305

- Reasoning behind the rename in referenced commit
2020-06-16 18:08:50 +12:00
Kevin Ansfield
73e14b4235 🐛 Fixed small images appearing smaller in post content
closes https://github.com/TryGhost/Ghost/issues/11913

- updated `@tryghost/kg-default-cards` with fixed `srcset` behaviour
  - does not output `srcset` unless we know the image's original size
  - does not output `w` values for sizes that are bigger than the original image
2020-06-15 16:24:02 +01:00
Kevin Ansfield
6535f2ef75 Fixed regression tests
refs 937df2c7ec

- some mobiledoc->html output with blank img alt attributes was missed in regression tests
2020-06-15 09:41:11 +01:00
Kevin Ansfield
937df2c7ec ️ Added blank alt attributes to images when no alt text is provided
no issue

- screen readers often resort to reading out the filename when alt text is not provided which is not useful. By adding `alt=""` (or just `alt`) to `<img>` elements the image is skipped by screen readers providing a smoother experience
- updated `@tryghost/kg-default-cards` which contains above change for `image` and `gallery` cards
2020-06-15 09:24:15 +01:00
Kevin Ansfield
f4d9a41d3b
Added ?paid query parameter to Admin API members browse endpoint (#11892)
no issue

- NQL does not support the relationship setup that members->stripe customer<->stripe subscriptions uses so it wasn't possible to use the `filter` param to query against having an active subscription
- adds `customQuery` bookshelf plugin that allows customisation of SQL query used in `findPage` method by individual models
- use `customQuery` in Member model to set up joins and conditionals to select free/paid members when `options.paid` is present
- allow `?paid` param through API and permitted options for member model
2020-06-12 12:12:10 +01:00
Kevin Ansfield
f1291058ec Fixed regression tests 2020-06-12 10:08:10 +01:00
Nazar Gargol
7904c303a7 Added invalid import record errors and counts
no issue

- This new format allows to return additional metadata with failed import records. The data for invalid records is returned in following format:
```
{
    count: {count_of_invalid_records},
    errors: [{
      message:	"Members not imported. Members with duplicate Stripe customer ids are not allowed." // message field of the error
     context:	"Attempting to import members with duplicate Stripe customer ids." // context field of the error
     help:	"Remove duplicate Stripe customer ids from the import file, and re-run the import." // help field of the error
     count:	2 // count of this specific error
    }]
};
- Errors are grouped by their context fields because message fields sometimes can contain unique information like Stripe customer id, which would produce too many errors in case of bigger datasets.
2020-06-12 19:59:36 +12:00
Kevin Ansfield
50846e310f Fixed post model regression test
no issue

- `html` output contains `srcset` since 70b205618f
2020-06-11 14:02:53 +01:00
Nazar Gargol
81fc5f8eda Added special handling for member import with Stripe connection
no issue

- When imported member contains stripe_customer_id data but there is no Stripe configured on the Ghost instance such import should faiil. The logic is consistent with one where import fails after not being able to find customer in linked Stripe account
- Fixed import stats to show import failures instead of "duplicate" when the validation error is of "Stripe" origin
2020-06-09 23:02:38 +12:00
Daniel Lockyer
d44dae0c79 Fixed wrong use of moment
no issue

- whilst updating moment, a test that was doing timezone calculation was
  failing. We were only importing moment before, and not moment-timezone
- this commit fixes that
2020-06-08 19:43:55 +01:00
Nazar Gargol
633ba27f0e Added custom label assignment to imported members
no issue

- There is a need to be able to label certain import group of members
with custom labels. This will allow to distinguish/filter these newly
imported  members.
- Allowed `POST /members/csv/` endpoint to accept `labels`
field parameter which assigns labels to every member from imported csv.
2020-06-05 16:22:09 +12:00
Nazar Gargol
12c8b63a4a Added more specific error handling when adding duplicate labels
no issue

- Similarly to other additive api methods  (e.g. members.add) returned more specific ValidationError with contex filled in with the reason why adding did not succed.
- This change is needed for more graceful label handling when adding new members through import
2020-06-05 00:23:10 +12:00
Fabien O'Carroll
37a1947fc5 Fixed broken regression tests from #11861
no-issue

The config is now a class and the context of `this` was not bound
2020-06-02 16:20:57 +02:00
Hannah Wolfe
dd9bf73a0b Updated edit URLs to use consistent correct URLs
- Ghost Admin has a redirect system built in, so not using the # does work
- However, the usage of # or no # was inconsistent, which is always wrong
- Using the correct Ghost Admin URLs makes it easier to reason about the behaviour
2020-06-01 08:42:11 +01:00
Vikas Potluri
c7127d5a93
Removed final non-destructured imports of core/server/lib/common (#11858)
* removed final non-destructured imports of core/server/lib/common
* fixed lint
2020-05-28 17:54:18 +01:00
Kevin Ansfield
35f8042d7b
Added ?search= param to Admin API members endpoint (#11854)
no issue

- adds `search` bookshelf plugin that calls out to an optional `searchQuery()` method on individual models to apply model-specific SQL conditions to queries
- updated the base model's `findPage()` method to use the search plugin within `findPage` calls
- added a `searchQuery` method to the `member` model that performs a basic `LIKE %query%` for both `name` and `email` columns
- allowed the `?search=` parameter to pass through in the `options` object for member browse requests
2020-05-28 10:14:02 +01:00
Vikas Potluri
15d9a77092
Moved config from server to shared (#11850)
* moved `server/config` to `shared/config`
* updated config import paths in server to use shared
* updated config import paths in frontend to use shared
* updated config import paths in test to use shared
* updated config import paths in root to use shared
* trigger regression tests
* of course the rebase broke tests
2020-05-27 18:47:53 +01:00
Vikas Potluri
c17a6e457c
Removed import of common sub-modules (#11849)
* replaced import of `common/logging` with destructuring the "module" in migrations
* removed reference to `index` in admin regression test
2020-05-26 21:12:13 +01:00
Kevin Ansfield
071ab9774b
Added Admin API endpoint for basic member stats (#11840)
no issue

- moves members stats generation for the admin graph from the client to the server
- outputs a basic totals count across a requested date range of 30, 90, 365 days, or all time. See below for the response shape
- leaves heavy lifting of the counts to the SQL engines - tested on a dataset of 100k members and query performance is <100ms

```
GET /ghost/api/canary/members/stats/?days=30

{
    total: 100000,
    total_in_range: 20000,
    total_on_date: {
        '2020-04-25': 19000,
        '2020-04-26': 19500,
        // continues until today's date
    },
    new_today: 200
}
```
2020-05-26 10:38:42 +01:00