Commit Graph

842 Commits

Author SHA1 Message Date
naz
392140cb36
Allowed for comped field when creating a member through Members API (#12278)
closes #12273

- `comped` field has been allowed when editing a member or importing from a CSV. There has been a usecase (Zapier Integration) for API client to create a member with "Complimentary" plan, which made this change necessary
- Previously the logic for comped field was to skip and continue member record creation if Stripe was not connected. Now we throw an error - same as the one we have been throwing before when stripe_customer_id field was passed in. The implication of this change is that we won't be creating any record now if comped === true and Stripe is disabled. 
- Bumped admin-api-schema-package. Contains `comped` schema change so this field gets passed through to controller
2020-10-14 13:24:09 +13:00
Kevin Ansfield
9cbeb74db0
🐛 Fixed broken embeds cards when pasting links to Wordpress sites (#12262)
closes https://github.com/TryGhost/Ghost/issues/12260

- if a card type was not explicitly chosen (i.e. a url was pasted into the editor) then abort fetching the oembed endpoint if we detect it's a `wp-json` oembed and return a bookmark card payload instead
- cleaned up an unused argument in the internal `fetchBookmarkData()` method
2020-10-06 08:44:03 +01:00
Naz
39e403d176 🐛 Fixed meta attributes calculation on post preview
closes https://github.com/TryGhost/Ghost/issues/12247

- Internal preview controller was lacking "mapping" call to post object which handled not only missing meta attribute information but lots of other mappings (e.g. users, tags, etc.)
- Have added a regression test to catch issues like this in the future
2020-10-05 17:52:40 +13:00
naz
071bf8edb5
Extracted Admin API JSON schemas into @tryghost/admin-api-schema (#12237)
closes https://github.com/TryGhost/Ghost/issues/10628

- JSON Schemas were extracted into a separate module to allow other clients to reuse them (for example documentation). Having them in a separate package also slims down the amount of code needed to be maintained in the core.
- Updated canary API input validators to use admin-api-schema module
- Removed canary schemas that moved into admin-api-schema package
- Updated v2 API input validators to use admin-api-schema package
- Removed v2 schemas that moved into admin-api-schema package
- Updated tests to contain needed information in apiConfig to pick up correct validation
- Added @tryghost/admin-api-schema package dependency
2020-09-30 16:25:12 +13:00
Fabien 'egg' O'Carroll
854a41e556
🐛 Fixed CSV import json-schema email validation (#12239)
no-issue

By using the "email" validation, we were validating emails in CSV
imports using a different validator to the rest of the API. AJV's built
in email validation was failing on emails with "special" characters,
such as letters with an umlaut above them.

This commit brings the validation for CSV imports in line with the rest
of the API.
2020-09-29 12:51:35 +01:00
Kevin Ansfield
1dfaf8c5f8
Added migration to store from/reply-to values in email table (#12204)
no issue

The email table should be a reference for all data that was used when sending an email. From and Reply-to addresses can change over time and we don't have any other reference for their value at the time of sending an email so we should store them alongside the email content.

- schema updated with `from` and `reply_to` columns
- both are set to `nullable` because we don't have historic data (can be populated and changed in later migrations if needed)
- neither `from` or `reply_to` have `isEmail` validations because they can have name+email in an email-specific format
- will help keep concerns separated in the future. `mega` service can deal with all of the email contents/properties, and the `bulk-email` service's concerns are then only email sending and any provider-specific needs
2020-09-24 08:20:10 +01:00
Naz
43153ba31e Disabled changing webhook's integration_id through PUT API
refs #12033

- Allowing to change parent integration opens up possible security holes and has no clear usecase at the moment. After a webhook record is created it should not be possible to change parent integration.
- Had do partially duplicate JSON schema definition from webhooks definition as there is no proper composition technique available in current version of JSON Schema.
2020-09-24 16:24:02 +12:00
Naz
af516e130c Fixed error message checks in members regression test
refs 6f1abc610a

- Additional period `.` was introduced in referenced commit which broke these tests
- The period was added to follow general convention of ending error messages with a perio (in some situations validation message didn't make sense without proper punctuation)
2020-09-24 15:31:19 +12:00
Naz
6f1abc610a Added check for parent integration_id when creating a webhook
refs https://github.com/TryGhost/Ghost/issues/12033
refs https://github.com/TryGhost/Ghost/issues/10567

- Creating a webhook without valid parent integration leads to orphaned webhook records, which shoult not ever happen
- This scenario is only possible for non-integration authentication,
because in case of integration being authenticated it's id is
automatically assigned to creatd webhook
2020-09-24 15:09:51 +12:00
naz
bbcc83dadb
Added support for ordering Post API resources by fields coming form posts_meta table (#12226)
refs #11729

- When ordering is done by fields from a relation (like post's `meta_title` that comes form `posts_meta` table), Bookshelf does not include those relations in the original query which caused errors. To support this usecase added a mechanism to detect fields from a relation and load those relations into query. 
- Extended ordering to include table name in ordered field name.  The information about the table name is needed to avoid using `tableName` within pagination plugin and gives path to having other than original table ordering fields (e.g. order by posts_meta table fields)
- Added test case to check ordering on posts_meta fields
- Added support for "eager loading" relations. Allows to extend query builder object with joins to related tables,
which could be used in ordering (possibly in filtering later). Bookshelf does not support ordering/filtering by proprieties coming from relations, that's why this kind of plugin and query expansion is needed
- Added note about lack of support for child relations with same property names.
2020-09-24 13:32:40 +12:00
Kukhyeon Heo
635d26469f
Fixed settings reinit to only emit changed settings events (#12184)
closes #12038

Previously we were emitting changed events for _all_ settings which would
cause any listeners for those to be triggered, this ensures that listeners are
only triggered if the corresponding setting, _did_ in fact change.
2020-09-23 14:35:03 +01:00
Fabien O'Carroll
171908cefa Added support for filtering Members CSV exports
no-issue

Adding these options to the endpoint will pass them to the model layer,
allowing us to filter members CSV exports
2020-09-23 14:29:15 +01:00
Fabien O'Carroll
800ffc2e8f Improved Members CSV export acceptance tests
no-issue

This updates the test to check the parsed CSV contains all the expected rows
2020-09-23 14:29:15 +01:00
Nazar Gargol
5766b39be6 Fixed failing regression tests
refs 32b37d7ba8

- Some of the regression tests were broken after changes in referenced commit
2020-09-22 16:13:57 +12:00
Nazar Gargol
32b37d7ba8 Improved error messaging for password reset process
refs #11878

- When password reset link is invalid previous messaging left the user
without clear information about why the reset failed and what they could do about it.
- Updated messaging around password reset tokens including detection of
when password token has invalid structure, has expired or has already
been used
2020-09-22 15:45:19 +12:00
Fabien 'egg' O'Carroll
812e4b682f
Added SingleUseToken model (#12215)
no-issue

This is a model for the tokens table, which handles the single use
aspect by customising the `findOne` method to automatically destroy the
model after reading from it
2020-09-18 15:05:56 +01:00
Fabien O'Carroll
a862a58b5f Updated the tokens table to have 32 char tokens
no-issue

After discussion with Matt, we decided that 192 bits for the token is a
good number, as it has no padding when base64 encoded and is more secure
than 128 bits, whilst still a managable size.
2020-09-17 17:33:43 +01:00
Fabien O'Carroll
01e62e3d79 Added tokens table to the database
no-issue

This is a table to store single use tokens for use in magic links, the
columns are as simple as possible at the moment and are designed as:
id - standard ObjectID like all of our tables
token - 128bit base64 encoded string
data - arbitrary data to store against the token
created_at - timestamp to allow for expiry to be implemented for tokens
2020-09-17 16:08:35 +01:00
Rish
7a3839fb14 Added new portal config flag
no issue

- The new Portal config flag allows switching on Portal conditionally with config
- The dev experiment flag still works for enabling Portal
- The flag currently defaults to `false` as Portal is still a beta feature and switched off by default
- We expose it on the admin api config endpoint so that the Ghost-Admin client can use it to conditionally render Portal settings
2020-09-17 19:40:34 +05:30
Nazar Gargol
bd16c2216a Fixed acceptance tests for posts_meta refactor
refs d9dbcf6147
2020-09-17 21:19:26 +12:00
Nazar Gargol
d177852d1f Fixed posts_meta relation in test fixtures
refs d9dbcf6147

- Fixes broken commit
2020-09-17 20:46:45 +12:00
Nazar Gargol
d9dbcf6147 Corrected data generator fixture with posts_meta fields
refs https://github.com/TryGhost/Ghost/pull/11102

- When splitting out posts_meta table out of posts table, the fixtures were not corrected.
- There were no acceptance/regression tests which involved posts_meta fields which is why the error was not detected
2020-09-17 19:34:36 +12:00
Nazar Gargol
e9204c9168 Updated invitations test coverage with more specific checks
refs https://github.com/TryGhost/Ghost/issues/11878

- Updated test cases in preparation to improve messaging in referenced
issue.
2020-09-16 14:24:24 +12:00
Kevin Ansfield
76c1b60a4d
Added schema+migration for email_{batches,recipients} tables (#12192)
no issue

We want to store a list of recipients for each bulk email so that we have a consistent set of data that background processing/sending jobs can work from without worrying about moving large data sets around or member data changing mid-send.

- `email_batches` table acts as a join table with status for email<->email_recipient
  - stores a provider-specific ID that we get back when submitting a batch for sending to the bulk email provider
  - `status` allows for batch-specific status updates and picking up where we left off when submitting batches if needed
  - explicitly tying a list of email recipients to a batch allows for partial retries
- `email_recipients` table acts as a join table for email<->member
  - `member_id` does not have a foreign key constraint because members can be deleted but does have an index so that we can efficiently query which emails a member has received
  - stores static copies of the member info present at the time of sending an email for consistency in background jobs and auditing/historical data
2020-09-14 12:21:58 +01:00
naz
cbdc91ce48
Added Location header to API's POST request responses (#12186)
refs #2635

- Adds 'Location' header to endpoints which create new resources and have corresponding `GET` endpoint as speced in JSON API - https://jsonapi.org/format/#crud-creating-responses-201. Specifically:
    /posts/
    /pages/
    /integrations/
    /tags/
    /members/
    /labels/
    /notifications/
    /invites/

- Adding the header should allow for better resource discoverability and improved logging readability
- Added `url` property to the frame constructor. Data in `url` should give enough information  to later build up the `Location` header URL for created resource.
- Added Location header to headers handler. The Location value is built up from a combination of request URL and the id that is present in the response for the resource. The header is automatically added to requests coming to `add` controller methods which return `id` property in the frame result
- Excluded Webhooks API  as there is no "GET" endpoint available to fetch the resource
2020-09-14 22:33:37 +12:00
Talha
afa976066a
🐛 Fixed table constraint error when updating member's email with an already existing email (#12178)
closes #12045

- When member's email is updated to an already existing email of different member it caused table's unique constraint error, which was not handled properly. 
- Added handling for this error similar to one in members `add` method.
2020-09-10 16:03:57 +12:00
Nazar Gargol
453bc2c491 Fixed regression test for integrity check
refs 5582d030e3

- When not touching this area for longer time always forge following:  `routes.yaml` configuration file in /content/settings comes as a copy of `default-routes.yaml` file from frontend/services/settings/
- Always remember to clean up junk "default" files in the content/settings folder to make things less confusing!
2020-09-10 12:57:03 +12:00
Nazar Gargol
a8202eb80d Fixed regression test for integrity check
no issue

- The file that is better suited for integrity check is the `*-default.yaml` config because it's the one that gets copied through when there is none or configuration is broken
2020-09-10 11:20:38 +12:00
Nazar Gargol
5582d030e3 Added routes.yaml content checksum storage to the db
closes #11999

- When the routes.yaml file changes (manually or through API) we need
to store a checksum to be able to optimize routes reloads in the future
- Added mechanism to detect differences between stored and current routes.yaml hash value
- Added routes.yaml sync on server boot
- Added routes.yaml handling in controllers
- Added routes hash synchronization method in core settings. It lives in core settings
as it needs access to model layer. To avoid coupling with the frontend settings it accepts
a function which has to resolve to a routes hash
- Added note about settings validation side-effect. It mutates input!
- Added async check for currently loaded routes hash
- Extended frontend settings loader with async loader. The default behavior of the loader is
to load settings syncronously for reasons spelled in 0ac19dcf84
To avoid blocking the eventloop added async loading method
- Refactored frontend setting loader for reusability of  settings file path
- Added integrity check test for routes.yaml file
2020-09-10 10:54:57 +12:00
Talha
c8a6939b2e
🐛 Fixed error caused by accepting invitation with existing email (#12172)
closes #12060

- A 500 error what happening when invited user provided an email that is associated with an existing user
- Additional validation for existing email address was added to prevent invalid data hitting db constraint error
2020-09-09 11:58:53 +12:00
naz
fcc6401445
Added routes_hash setting (#12171)
refs #11999

- The `routes_hash` setting will be used during the boot process to update the hash
of currently loaded routes.yaml file in case it's different from last restart
2020-09-07 21:54:55 +12:00
Kevin Ansfield
936d7a315a 🐛 Fixed incorrect status used for trial subscription query
refs 96c18edc82

- replaced `trialed` with `trialing` to correctly match the status used in Stripe
2020-09-02 13:07:48 +01:00
Kevin Ansfield
96c18edc82 🐛 Fixed members with Stripe trials not receiving paid-members newsletters
no issue

- members who have trial subscriptions added directly via Stripe will have a status of `"trialed"` in their Ghost subscription
- the `paid: true` filter was not taking that into account meaning trial users were not receiving newsletters sent to paid members even though they have a "paid" subscription
2020-09-02 00:34:26 +01:00
Rishabh Garg
bc2f6eb8e9 Added migrations for support and reply email address setting (#12163)
no issue

- Added default settings for the two new setting fields - `members_support_address` and `members_reply_address`
- Added migrations for setting group for new email settings
- Migration sets current from address as new support address default
- Added migration to set new support address same as from address
- Updated tests for new settings
- `members_support_address` - How members can reach for help with their account, public setting
- `members_reply_address` - Where you receive responses to newsletters
2020-08-31 17:16:53 +05:30
Nazar Gargol
0f81e5e84a Fixed regression tests for Tags APIs
refs #12167

- The reason for failing tests was  https://github.com/TryGhost/Ghost/issues/6104 . As we already test for ordering elsewhere, didn't include more "if/else" logic in tests just for the sake of passing. Refactored them to be order-independent instead
2020-08-31 21:09:12 +12:00
Rish
8cf9dc05e3 Revert "Added migrations for support and reply email address setting (#12163)"
This reverts commit 2216019a86.
2020-08-31 14:25:29 +05:30
Nazar Gargol
eb8c0fcff9 🐛 Fixed Tags API v2 to return posts count
closes #12167

- Tags API v2 was ignoring `count.posts` include parameter.
- Regression was introduced with a3f693b472
- Introduced regression tests across all Content API versions to avoid similar bug in the future
2020-08-31 18:46:35 +12:00
Nazar Gargol
791bcc6adb Improved tests for "include=posts.count" query parameter
refs #12167

- Updated tests to check for concrete post counts numbers. This would allow catching more regression bugs in the future
- Made sure check is performed consistently in tests where posts.count
parameter is present
2020-08-31 17:30:21 +12:00
Rishabh Garg
2216019a86
Added migrations for support and reply email address setting (#12163)
no issue

- Added default settings for the two new setting fields - `members_support_address` and `members_reply_address`
- Added migrations for setting group for new email settings
- Migration sets current from address as new support address default
- Added migration to set new support address same as from address
- Updated tests for new settings
- `members_support_address` - How members can reach for help with their account
- `members_reply_address` - Where you receive responses to newsletters
2020-08-27 20:27:50 +05:30
Nazar Gargol
bcca59ffbe Fixed regression tests for members importer
refs 262d33b73c

- Added missing test changes from referenced commit
2020-08-27 19:28:18 +12:00
Kevin Ansfield
a995e9cc89 Allowed custom and search queries to be used with .findAll()
no issue

- extract filtering of an collection into a separate function
- use extracted function in `findAll()` so that it's query behaviour matches `findPage()`
2020-08-27 01:09:07 +01:00
Kevin Ansfield
5e64f113d5 Skipped separate count query in .findPage() for limit="all" requests
no issue

- for large result sets or complex queries the count query itself can be quite time consuming
- when `limit: 'all'` is passed as an option there's no need to perform a separate count query because we can determine the pagination data from the final result set
- skipped count query when `limit: 'all'` option is present
- re-ordered comments to be closer to the code they reference (ie, why we have our own count query instead of Bookshelf's `.count()`
2020-08-27 01:09:07 +01:00
Nazar Gargol
9d65de4b10 Removed created import labels when no records were imported
no issue

- When no members are succesfully imported through CSV import process the import label should not be created. Otherwise after multiple failed attempts to import there are orphaned labels in the system
2020-08-25 19:23:05 +12:00
Nazar Gargol
b8197023b3 🐛 Fixed ignored created_at and subscribed fields in Members API
closes #12156

- When adding new member through `POST /members` API or importing members with CSV importer `POST /members/upload` API created_at and subscribed were ignored
- Similar problem but only with `subscribed` field was present in `PUT /members/:id` API
- The regression was introduced with a bump of @tryghost/members-api to 0.26.0, specifically this change in upstream - a28bcc5b2a (diff-3daeef67d07a2a0f94c89a86cafcede9R44)
- Bumped @tryghost/members-api package to 0.28.2 fixing the underlying issue - 7b5f2e3cb7
2020-08-24 19:35:20 +12:00
naz
3a594ce22e
Improved error handling for batch inserted member records (#12146)
no issue

- When batch insert fails handling should be more granular and aim to retry and insert as many records from the batch as possible.  
- Added retry logic for failed member's batch inserts. It's a sequential insert for each record in the batch. This implementation was chosen to keep it as simple as possible
- Added filtering of "toCreate" records when member fails to insert. We should not try inserting related members_labels/members_stripe_customers/members_stripe_customer_subscriptions records because they would definitely fail insertion without associated member record
2020-08-20 17:41:47 +12:00
Nazar Gargol
6512b98049 Updated flaky member import tests
no issue

- Because there's concurrency involved during member import the order cannot be guaranteed.
2020-08-19 16:43:58 +12:00
Rishabh Garg
2bac2c67cc
Granted Admin users ability to impersonate member (#12132)
refs #12126

- Adds migration to add impersonation permission to administrators
- Adds default permission fixture to allow administrators to read member impersonation urls
- Allows administrators to create member impersonation magic links
2020-08-18 14:17:56 +05:30
naz
51c9a50c4f
Added validation layer to members import endpoint (#12131)
no issue

- Additional validation is needed for imported data because in case of bulk insertions (through knex) we bypass model layer validation - this could lead to invalid data in the database, which would be hard to fix.
- Chose validation method we use for other endpoints - through JSON Schema. It proved to be very performant (200ms overhead for 50k records). When comparing it with iterative method (validating each record separately) this was adding about 17s of overhead.
- Refactored returned values from "sanitizeInput" method to encapsulate more logic so that the caller doesn't have to calculate amount of invalid records and deal with error types
- Whole sanitizeInput method could now be easily extracted into separate module (somewhere close to members importer)
- Bumped members-csv package. It is meant to handle empty string values - '' and null, which should allow validating member records more consistently!
2020-08-17 23:28:57 +12:00
Nazar Gargol
ef41c57974 Added test coverage for invalid member import emails
no issue

- This test should serve as a control for new batched importer
2020-08-13 22:14:52 +12:00
Kevin Ansfield
c434666ba2 🐛 Removed [http://url/] output in member email preview text
no issue

- we output the post excerpt in a hidden div in the email template so that email clients pick it up as the "preview" text when listing emails
- when no custom excerpt is provided the preview text is grabbed from post.excerpt which is the first 500 chars of the post.plaintext value
- post.plaintext formats links as "Link [http://url/]" which is unwanted in html email previews

- add a basic replacement to the post email serializer to remove any `[http://url/]` occurrences from the post excerpt before rendering the email content
2020-08-12 20:14:06 +01:00
Kevin Ansfield
163092f377 🐛 Fixed apostrophes not displaying correctly in Outlook for member emails
refs https://github.com/TryGhost/Ghost/issues/11536

- Outlook supports `&#39;` as a special char for apostrophes but not `&#apos;` which is what cheerio/juiced render
- adds a basic string placement to the email serializer to switch to the older style of special char
2020-08-12 20:01:42 +01:00
Fabien 'egg' O'Carroll
1294e3f92c
Replaced all usage of member models with members-api (#12117)
no-issue

* Added stripeSubscriptions relation to member model

This allows us to fetch the subscriptions for a member via standard
model usage, e.g. `withRelated: ['stripeSubscriptions']` rather than
offloading to loops and `decorateWithSubscriptions` functions, this is
more performant and less non-standard than the existing method.

* Updated serialize methods to match existing format

The current usage of `decorateWithSubscriptions` and the usage of
members throughout the codebase has a subscriptions array on a stripe
object on the member, this ensures that when we serialize members to
JSON that we are using the same format.

There is definitely room to change this in future, but this is an
attempt to create as few breaking changes as possible.

* Installed @tryghost/members-api@0.26.0

This includes the required API changes so that everywhere can use
members-api directly rather than models and/or helper methods
2020-08-12 14:17:44 +01:00
Nazar Gargol
597bc12088 Fixed tests with members labels
refs 7d3f6e32ca

- Missed to commit fixture change
2020-08-12 17:02:08 +12:00
Nazar Gargol
7d3f6e32ca Addes sort order support for imported members
no issue

- Member's labels have to have sort_order assigned when added/edited. This was lacking from batched importer.
- Implementation is based on logic used in model's base - e484709e73/core/server/models/base/index.js (L81-L86)
2020-08-12 16:34:15 +12:00
Daniel Lockyer
5b471e1bbe Extracted promise libs and history into @tryghost/promise
- deleted files under `core/server/lib/promise` and related test files
- added `@tryghost/promise` as a dependency
- fixed all local requires to point to the new package
2020-08-11 18:44:21 +01:00
Daniel Lockyer
c9a5b28669 Extracted core/server/lib/security to @tryghost/security package
- code and tests were extracted out to this package
- deletes these files
- replaces all local requires, and adds it as a dependency
2020-08-11 14:06:50 +01:00
Daniel Lockyer
8799feb801 Replaced constants file with @tryghost/constants
- extracted constants file into a new package
- replaced all local requires of the file with new package
2020-08-11 12:51:16 +01:00
Hannah Wolfe
e84621d6ef 🐛 Readded missing server.start event
closes #12118

- server.start was mistakenly removed in 71f02d25e9
- it is used for loading themes (and other things) and is critical
- added tests to prevent this regressing again in future
2020-08-11 10:09:24 +01:00
Hannah Wolfe
71f02d25e9 Refactored server announce functions to be clearer
- the announce functions exist for the purpose of communicating with Ghost CLI
- the functions were called announceServerStart and announceServerStopped, which implies they tell Ghost CLI when the server starts and stops
- however, the true intention / purpose of these functions is to:
    - either tell Ghost CLI when Ghost has successfully booted (e.g. is ready to serve requests)
    - or tell Ghost CLI when the server failed to boot, and report the error so that Ghost CLI can communicate it to the user
- therefore, I've refactored the old functions into 1 function to make it clearer they do the same job, but with 2 different states
- also added some tests :D
2020-08-09 17:25:15 +01:00
Fabien 'egg' O'Carroll
d15446593a
Added support for ON DELETE CASCADE to the schema (#12105)
no-issue

We are in the process of creating migrations to add foreign key constraints
and cascading deletes to the members_stripe_* tables to make listing members
and deleting members faster. As well as the migrations we need to update the
database schema so that new installations have the correct indexes and constraints.

Co-authored-by: Kevin Ansfield <kevin@lookingsideways.co.uk>
2020-08-05 13:20:30 +02:00
Nazar Gargol
60ae9e82f9 Fixed integration_id assignment for webhook when creating through API key auth
refs 173e3292fa

- The bug was initially introduced in referenced commit. When request is done with `api_key` context, there should always be an `integration` object associated with it - 71c17539d8/core/server/services/permissions/parse-context.js (L36) . An `id` from `context.integration` not `context.api_key` has to be assigned to newly created webhook!
- The webhooks API is about to be declared stable in upcoming release, so no migration will be done
2020-08-04 16:43:24 +12:00
Nazar Gargol
1b449f4f53 🐛 Fixed 500 error in webhooks API when modifying non-existing webhooks
closes #12064

- Handled permission check bug by returning 404, same way it is returned in other permissions related places when handling non-existing resource. Example - 60907a7ae4/core/server/models/relations/authors.js (L355-L358)
2020-08-03 23:08:47 +12:00
Kevin Ansfield
0c5c7b32b8 Fixed members import regression tests
refs 14c6968388

- updates expected member labels length asserts to account for auto-generated import label being attached
2020-07-26 18:57:11 +01:00
Kevin Ansfield
14c6968388 Fixed auto-generated label not being attached to members during import
no issue

- label was being created and returned in the response OK but it was missing from the set of labels that get added to members during import
2020-07-26 18:30:58 +01:00
Fabien O'Carroll
c46475f5be Supported optionally cancelling subscriptions on delete
no-issue

This updates the Admin API Member resource to *not* cancel subscriptions
by default, and adds a `cancel` option. This can be used over HTTP by
including a `cancel=true` query parameter.
2020-07-24 16:01:06 +02:00
Fabien 'egg' O'Carroll
5144a0e09c
Updated Member model to cascade on destroy (#12077)
no-issue

Up until now we have left orphaned rows in members_stripe_* tables when
a member is deleted, this updates the destroy method so that we cascade
and remove any MemberStripeCustomer and StripeCustomerSubscription
models related to the Member.

This also adds regression tests for the new functionality as well as to
confirm the existing functionality of cascading to the members_labels
join table

This adds the relations of Subscription->Customer & Customer->Member
2020-07-23 18:21:10 +02:00
Kevin Ansfield
6504b0f6c3 Fixed settings unit tests
refs e04f55cce3

- added `tracker.uninstall()` so that previously set up `tracker.on()` listeners are not called by later tests
- fixed `emits edit events` test which was not correctly mocking the select and update queries
2020-07-22 22:12:00 +01:00
Kevin Ansfield
62c72cb58b Fixed oembed test
refs e04f55cce3

- added `nock.cleanAll()` so that there is no inter-test dependencies
- the failing test was successfully passing previously due to mocha's retry behaviour eventually exhausting nock request handlers that were set up in other tests and intended not to be called
2020-07-22 19:49:30 +01:00
Kevin Ansfield
8ceabbcfba Added automatic labelling of all members added during an import
no issue

Having all members created during an import labelled with a specific "import label" is useful for later operations such as bulk delete/edit or simply recording how and when a member was created.

- automatically create a label with the date/time the members CSV import occurred and assign it to all imported members
- return the import label data in the API response so that clients can react accordingly such as automatically filtering the members list by the label once an import finishes
2020-07-22 17:52:43 +01:00
Daniel Lockyer
415696a3f9 Replaced removed nock method
no issue
2020-07-21 20:25:06 +01:00
Hannah Wolfe
995a4eb46e 🐛 Fixed edit link for staff members
closes #12069

- the url in admin changed from team to staff, and wasn't updated in the frontend
2020-07-21 12:28:50 +01:00
Fabien 'egg' O'Carroll
6232981be7
🐛 Fixed importing Stripe Plans with amount 0 (#12062)
closes #12049

Stripe plans used to default to 0, and our new validation of plan
amounts were causing issues when importing from an older version of
Ghost, this updates the validation to be skipped when importing.

- Added regression test for importing plans
2020-07-20 14:59:23 +02:00
Nazar Gargol
a520cdad0b Added JSON Schema validations to Webhooks Admin API v3
closes #12033

- Added webhooks schemas and definitions.
- Added validation checking if integration_id is present when using session auth. This is needed to prevent orphan webhooks.
- Integrated webhook schemas into frame's validation layer.
- Added isLowerCase ajv keyword support. This is needed to be able to do isLowerCase validation using JSON Schema for webhooks.
2020-07-17 17:37:14 +12:00
Fabien 'egg' O'Carroll
8f660c3259
Improved settings validation (#12048)
closes #12001

* Moved settings validation to the model

This moves the settings validation out of the validation file and into
the model, as it is _only_ used there.

It also sets us up in the future for custom validators on individual
settings.

* Improved validation of stripe_plans setting

- Checks `interval` is a valid string
- Checks `name` & `currency` are strings

* Moved stripe key validation into model

The stripe key settings are all nullable and the regex validation fails
when the input is `null`. Rather than reworking the entirety of how we
validate with default-settings validation objects, this moves the
validation into methods on the Settings model.

* Added tests for new setting validations

Adds tests for both valid and invalid settings, as well as helpers
making future tests easier and less repetitive
2020-07-15 17:11:27 +02:00
Fabien O'Carroll
48eca5f618 Added amp_gtag_id setting migration & default
refs #11980

This lays the ground for the rest of the work surrounding Google
Analytics in AMP.
2020-07-15 13:43:25 +02:00
Nazar Gargol
ced3e28d60 🐛 Fixed cache invalidation for published posts with no content
closes #12015
refs 95880dddeb

- The bug was caused by falsy plaintext field assignment to empty string `''` when the html content was `null`. Because of the `setEmptyValuesToNull` function (referenced commit), there is no sense to assign empty string value to plaintext property, because it would still end up being `null`
- The `''` -> `null` conversion was confusing the model layer to think that some fields were changed, where in reality none did. This in turn lead to a bug with falsy cache invalidation
2020-07-15 17:41:24 +12:00
Nazar Gargol
a75e1e27db 🐛 Fixed cache invalidation when editing post_meta fields
closes #12016

- The change detection didn't work when editing post_meta fileds because we only check current model's `_changed` fields when performing `wasChanged()` check
- A solution was adding change tracking of post_meta relation to currently edited post model and overloading `wasChanged` method to check these fields as well
2020-07-14 23:48:36 +12:00
Fabien O'Carroll
06c7152f4b 🐛 Fixed Stripe webhooks for subdirectory setups
no-issue

- Added breaking test for webhook url including subdirectory

- Previously the webhook handler URL was generated incorrectly when
  running Ghost on a subdirectory, appending the path to the root of the
  host, this fix ensures that the subdirectory is included before the
  path.
2020-07-10 17:43:46 +02:00
Fabien 'egg' O'Carroll
20a95de8f7
Added new stripe webhook settings (#12028)
no-issue

They will be used to store webhook information so that we can persist it between
boots and simplify the creation process of webhooks in members
2020-07-10 16:26:06 +02:00
Rish
d6f4f01eff Added new settings integrity tests
refs https://github.com/TryGhost/Ghost/issues/12026

- We made changes to settings table structure in 3.22 which added new columns for `group` and `flags`
- Any new setting needs explicit group and flag migrations since or they fallback to default group value of `core`
- This tests ensures
  - hash is updated in DB integrity test anytime default-settings are changed
  - that a migration is present by maintaining a whitelist of allowed core settings and failing if new setting is added without correct group migration
2020-07-10 18:32:54 +05:30
Nazar Gargol
d6267340a1 Updated Content API resource ordering to be same as slugs in filter
closes #11994

- Adds support for ordering based on slug filter  that contains a slug-is-in filter. It is applied only to Content API's resources - post, page, tag, author. The order is applied in the same order in which slugs appear in the filter.
- For, example providing following  query parameter filter for any of the above resources: `?filter=slug:[kitchen-sink,bacon,chorizo]`, would filter them by these slugs and order in the same way defined in the filter
- Can be used in handlebars templates in following way: `{{#get "tags" filter="slug:[slugs,of,the,tags,in,order]"}}`
- The property conteining this new order is assigned to `autoOrder` instead of `rawOrder` intentionally. This explicit asstignment would allow distinguishing where the 'orderRaw' comes from the model or the API layer. Apart from  adding necessary context this separation makes it easier to refactor separately model layer and API specific ordering in the future
- This commit also fixes default filtering for `author` resource in Content API. The serializer was never used before as it was missing from `serializers/index.js` module.
2020-07-10 18:33:00 +12:00
Fabien O'Carroll
a607610c3a Added new meta columns to tags table
no-issue

This prepares the database for future work, allowing tags to store metadata
2020-07-08 11:49:22 +02:00
Nazar Gargol
173e3292fa Fixed integration_id handling in Webhooks Admin API
no issue

- Changes introduced to both API v3 and v2
- Makes sure to use the same integration_id as authenticated integration for the webhook's data.
-  Makde it is impossible to create orphaned webhooks using token authentication
- Allowed only parent integration to edit it's children webhooks. Throwing permission error otherwise
2020-07-08 16:54:31 +12:00
Nazar Gargol
3c21b45ad8 Fixed linting error
refs 256b407dd2
2020-07-07 21:07:16 +12:00
Nazar Gargol
256b407dd2 Refactored webhook acceptance test suite
no issue

- Refactored a test as it was using outdated `done()` convention instead of relying on promises
- Removed linting warnings for variable shadowing
- This is precursor work for webhook regression test updates needed in near future work
2020-07-07 21:02:11 +12:00
Rish
c0e3c8bcfe Added new settings for portal button customization
no issue

- Adds new portal settings - `portal_button_style`, `portal_button_icon` and `portal_button_signup_text`
- New settings allows customization of portal button
- Updates tests to include new settings
2020-07-07 14:06:04 +05:30
Hannah Wolfe
8c8af3b4ac Fixed additional bracket in fixture code example
closes #11908

- The post publishing-options shows a snippet of JSON-LD in a code block, that had an extra comma and curly bracket
2020-07-06 19:49:10 +01:00
John O'Nolan
70f16f3d97
🎨 New Google AMP template (#11986)
- Refreshed the design of the default Ghost AMP template
- Updated tests
2020-07-06 17:09:24 +01:00
Daniel Lockyer
8956a9df35 Merged 3.22.2 into master
* tag '3.22.2':
  v3.22.2
  Updated Ghost-Admin to v3.22.2
  Emitted all settings events on reinit of cache (#12012)
  🐛 Updated access to be true by default in v3 API
  Hardened members subscription migration against missing data (#12009)
2020-07-06 16:19:55 +01:00
Hannah Wolfe
289c1b3e8a 🐛 Updated access to be true by default in v3 API
closes #11990

- access should be a members feature, but it was already accidentally exposed to the theme layer
- it has now been added to the API even if members is disabled
- access defaults to true, unless members is enabled
- when members is enabled, access is set to the currently logged in members' access
2020-07-06 16:04:29 +01:00
Daniel Lockyer
f65c425786 🐛 Fixed searching for free/paid members
no issue

- when searching for paid/free members, the `members_stripe_customers`
  table would be joined into the query on `members`
- this table also has a `name` and `email` field, so both MySQL and
  SQLite would complain about ambiguous fields in the query
- the result of this would be a 500 error thrown inside Ghost, and no
  useful response to the user
- this commit explicitly chooses the `members` table to check against,
  and also adds a test for this
2020-07-06 09:09:08 +01:00
Fabien 'egg' O'Carroll
37d577fa90
Remove references to bulk email settings (#12000)
refs #10318

* Ignored bulk_email_settings when importing
* Removed custom handling for bulk_email_settings
* Removed bulk_email_settings from default-settings
* Ignored bulk_email_settings when editing settings
2020-07-03 16:30:54 +02:00
Fabien 'egg' O'Carroll
9df217b04e
Added mailgun_{domain,api_key,base_url} settings (#11992)
refs #10318

 - Adds new default settings
 - Adds migration for populating settings with bulk_email_settings data
 - Fixes regression tests
2020-07-03 11:00:20 +02:00
Kevin Ansfield
e9ae0d9baa 🐛 Fixed srcset and sizes attributes being rendered when image resizing unavailable
closes https://github.com/TryGhost/Ghost/issues/11944

- updates `@tryghost/image-transform` to version that exposes `canTransformFiles()` which checks for `sharp` availibility
- updates `@tryghost/kg-default-cards` to version that accepts a `canTransformImage()` method as an option
- updates our `mobiledoc` lib to pass a `canTransformImage()` function that returns false if sharp is unavailable, the image extension is not supported, or the storage engine in use does not support image transforms
- updates `populateImageSizes` to fetch image sizes when transforms are unavailable as the render/not-render is now handled in the renderer and we don't need to worry about adding size information to the mobiledoc source
2020-07-02 18:03:22 +01:00
Nazar Gargol
6902f86b40 Added field mapping support for members import CSVs
no issue

- This is a precursor work allowing client to send csv to JSON
field mapping hash as a `mapping` field parameter in the incoming
request.
2020-07-02 15:18:39 +12:00
Kevin Ansfield
ec6a46a183 Updated tests that were stubbing old ghost_head/ghost_foot settings keys
refs https://github.com/TryGhost/Ghost/issues/10318
refs 9d7665bb43

- broken helpers weren't picked up previously because the tests were stubbing the old keys
- once the helpers were fixed the tests started failing because they weren't stubbing the new key
2020-07-01 17:58:12 +01:00
Kevin Ansfield
fd4a011995
🐛 Fixed settings cache being out of sync after migrations (#11987)
refs https://github.com/TryGhost/Ghost/issues/10318

- re-initialize settings cache after migrations by shutting down to clean up event listeners then and calling `init` again
- important to ensure `db.ready` event is not emitted until settings have finished re-initializing to avoid problems with background processes using the db connection which is disconnected/re-connected or being kicked off with out-of-date settings
2020-07-01 17:16:57 +01:00
Nazar Gargol
0ee92639dd Fixed member import test
refs 05f6faf846

- The copy for these errors has changed, same should've been done in the tests.
2020-07-01 17:42:58 +12:00
Kevin Ansfield
9810c41dca 🐛 Fixed facebook/twitter images having relative URLS in API output
no issue

Output serializer's url util was expecting `og_image` and `twitter_image` to be top-level attributes in the `attrs` object but they are actually nested under `posts_meta`.

- updated the code to use lodash's `get/set()` so that we can work with paths for easier handling of nested objects
- fixed unit tests where the mocked data under test did not match real-world data
2020-06-30 16:40:34 +01:00
Jonas Fabisiak
58ac033b97
Updated head to only output custom favicons (#11949)
closes #11932

- as per the issue, there is no need to output a <link> tag if the favicon is the default /favicon.ico, as all browsers automatically check for this 
- instead the favicon <link> is only output if a custom favicon has been set, telling the browser to look somewhere different to its default location
- some of the tests expected 3 links in the html head, but through the favicon change there are often only 2.

Co-authored-by: RenCloud <rencloud@pop-os.localdomain>
2020-06-30 15:07:08 +01:00
Hannah Wolfe
fa91c6c954
🐛 Moved access to an API property (#11967)
closes #11574

- the current implementation of the access property has it frontend only, and wired up only in one place
- this leaves it only available in a handful of places, e.g. can't use it in a post loop or get helper
- the current implementation also fails logically if the html content of the post is blank

This fix moves the behaviour to the API

- this ensures the field is always available no matter what context you are in
- it also updates the logic to use the same membersHasAccess logic as is used to gate the post, so it's always correct

TODO: should reconsider the location of this code
2020-06-30 14:46:46 +01:00
Rish
d5f68dbbc5 Updated serializers/model layer validation using settings type
refs https://github.com/TryGhost/Ghost/issues/10318

- Updates `boolean` serialization in v2/canary serializers to apply only for `boolean` type settings
- Updates `boolean` transformation in model layer `format`/`parse` to check on `boolean` type setting
- Removes error thrown on Read-only setting for settings edit endpoint
- Updates v2/canary input serializers to remove any Read-only settings (using RO flag) to avoid edits
- Added type/group mappings in the importer when pre-migration settings table import data is present
- Updates tests
2020-06-30 17:38:36 +05:30
Fabien 'egg' O'Carroll
ee786aaa5d
Cleaned up members & stripe settings (#11957)
* Updated members default settings

ref #10318

This pulls out the members_subscription_settings & stripe_connect_intgration settings into separate keys

* Updated usage of members_from_address

* Updated stripe_connect usage

* Updated members config to use new settings

* Updated members middleware to use isStripeConnected

* Updated members service to reload correctly

We reload the members-api instance when the related settings change, so
this makes sure we're listening to the correct settings changes

* Updated ghost_head helper to use new settings

* Updated theme middleware to use new settings

* Renamed members_allow_signup -> members_allow_free_signup

* Fixed tests after settings refactor

* Removed  from direct key settings key

* Fixed regression tests for settings api
2020-06-29 16:22:42 +02:00
Kevin Ansfield
f45198245f Updated Settings.populateDefaults() to account for available columns
refs https://github.com/TryGhost/Ghost/issues/10318

`Settings.populateDefaults()` is run before migrations during Ghost's startup. This can cause problems when new settings table columns are added (and populated in `default-settings.json`) because `populateDefaults()` was using the model layer which assumes that those columns are available, resulting in `ER_BAD_FIELD_ERROR: Unknown column` type errors.

- query the database for the available `settings` table columns
- switch to using raw knex queries without Bookshelf for insertions so that we're in control of the columns that are added
- use `_.pick` to skip any properties in `default-settings.json` that do not match to an available column - those columns will be added and populated by later migrations
- moving away from using the model to insert settings has the side-effect of not emitting `settings.added/edited` and `settings.x.added/edited` events, this should be fine because `populateDefaults()` is called before anything else is set up and listening
- added a call to `populateDefaults()` in our knex-migrator "before migration" hook so that we have consistent db state across both startup initialised migrations and manually triggered knex migrations
2020-06-29 13:22:10 +00:00
Fabien O'Carroll
ec119c458c Added migration utils for permissions
no-issue

These utils are the first steps toward getting the models out of our
migrations! The utils here interact directly with the database and where
possible don't reinvent the wheel, by using smaller building blocks to
build more comples ones.
2020-06-25 19:15:25 +02:00
Nazar Gargol
91e3630f36 Updated settings API v3 tests to check for correct types returned for specific keys
refs https://github.com/TryGhost/Ghost/issues/10318
refs 476fca6e5b

- Symetric change to one done in referenced commit which maps fields for API v2
2020-06-25 17:36:10 +12:00
Nazar Gargol
476fca6e5b Updated settings API v2 tests to also check for correct types
refs https://github.com/TryGhost/Ghost/issues/10318

- Added additional check for correct "type" fields in settings responses
- The list is based on pre-migration response from API v2, so makes it very clear there is no breaking change after the migration
2020-06-25 17:13:24 +12:00
Nazar Gargol
e394b5ad9c Added naive settings type options parameter support to settings API v2
refs TryGhost/Ghost#10318
refs 8fc526ff6

- This is symetric change to one done for v3 API (commited as 8fc526ff6)
- Added 'core' filtering for v2 API controller
2020-06-25 16:32:16 +12:00
Nazar Gargol
8fc526ff6e Added back compatible support for renamed settings types
refs https://github.com/TryGhost/Ghost/issues/10318
refs 1dc0405803

- Adds 1:1 mapping for filtering options  to renamed settings "type" to "gorup"
- Ignores the name changes and any old types
- Detailsed type -> group mappings can be checked in the refereneced migration commit
2020-06-25 00:55:40 +12:00
Kevin Ansfield
f08b5425e0 Fixed regression tests
no issue

- adds group and flags to the expected properties list for settings API responses
2020-06-24 12:14:06 +01:00
Kevin Ansfield
4a9e57c170
Added migration to add settings.{group,flags} columns (#11951)
refs https://github.com/TryGhost/Ghost/issues/10318

- `group`
  - to replace the `type` column, provides a more descriptive name for the columns use
  - for existing sites it will be populated by migrating data from the `type` column in a later migration
  - for new sites a minimal update has been added to `parseDefaultSettings()` to populate the `group` field when settings are created during startup - fixes the NOT NULL constraint on `settings.group`
- `flags`
  - signifies special handling that is different to other settings in a group
  - eg, `PUBLIC,RO` would indicate that the setting is available via unauthenticated endpoints and is read-only
2020-06-24 11:58:15 +01:00
Nazar Gargol
8f1905990d Added regression test coverage for editing deprecated settings
refs #10318
refs 118b7033e0

- Adds test coverage for fix introduced in referenedc commit
2020-06-24 22:40:57 +12:00
Nazar Gargol
f415389829 Updated admin admin v3 API test suite to match symmetric one in canary
no issue

- There are no functional differenec between canary and v3 APIs at the moment, thus these test suites should be kept in sync
2020-06-24 20:04:34 +12:00
Nazar Gargol
a21941ba51 Added settings ?type parameter to API regression tests
refs https://github.com/TryGhost/Ghost/issues/10318

- At the moment there were no test checking correct behavior of `?type` parameter. Given ongoing settings refactoring work expanded tests with some essential coverage
- Moved one of the test cases to be consistent with order in other suites
2020-06-24 20:01:28 +12:00
Nazar Gargol
b00369ac67 Fixed settings tests
refs 8a817050c5

- This change reflects field reshuffle done in referenced commit
-  NOTE: removed `members_email_auth_secret` key from respnse body validation as it seems like it should never have been exposed in the first place! This needs verification as it's a breaking change theretically
2020-06-24 14:55:15 +12:00
Nazar Gargol
8ea245fc7c Fixed importer mapping for renamed default_locale, active_timezone, ghost_* settings
refs #10318
refs 2614565d5a

- Adds importer mapping for fields as in referenced migration
- The intention is to allow exports from v2/v3 to still be compatilbe with current version
- Adds a mapper for deprecated fields and imports them with new settings "keys"
2020-06-24 14:19:52 +12:00
Nazar Gargol
f0a811e9fe Renamed ghost_head/ghost_foot to codeinjection_head/codeinjection_foot
refs #10318
refs 2614565d5a

- Renamed ghost_head/ghost_foot in settings to match the new names
introduced in migrations
- Above change lead to reshufling in the mappings in input/output
serializers
- Makes sure change is compatible with v2 API
2020-06-24 14:18:13 +12:00
Nazar Gargol
5f1060a8bf Renamed settings keys active_timezone to timezone
refs https://github.com/TryGhost/Ghost/issues/10318
refs 2614565d5a

- Renames to match referenced migration renames
- Fixed API responses so they are consistent with newly renamed fields
- Not returning lang and timezone keys from settings in API v2 ther rest should be returned in API v3/canary
2020-06-24 14:11:20 +12:00
Nazar Gargol
c4dff2d50e Renamed settings keys from default_locale to lang
refs #10318
refs 2614565d5a

- Renames default_locale settings key as introduced in referenced
migration
2020-06-24 14:07:24 +12:00
Kevin Ansfield
8a817050c5
Organised default settings into new groups (#11952)
refs https://github.com/TryGhost/Ghost/issues/10318

- precursor to migrating from `settings.type` to `settings.group`
- renames `blog` type to `site`
- renames `bulk_email` type to `email`
- moves settings out of `site` (previously `blog`) into more appropriate groups such as `core` or individual feature groups
2020-06-23 20:49:08 +01:00
Kevin Ansfield
0d82b3f92e Un-skipped populateImageSizes subdir test
refs 19c5e94259

- deleting `urlUtils` from the require cache caused problems in later unit tests that were stubbing `urlUtils` because they had required the `urlUtils` instance that existed before it got deleted/re-initialised in this test, so their modules under test ended up using non-stubbed instances
- fix by stubbing the `urlUtils` methods in this test too (we should probably fix this properly at some point so that config changes correctly propagate to `urlUtils` and we don't need to stub internals)
2020-06-23 15:54:58 +01:00
Kevin Ansfield
19c5e94259 Fixed unit tests interdependency issue
no issue

- there is a test interdependency issue between the `populateImageSizes` tests and routing tests when we clear the cached `url-utils` require
- commented the cache clear and subsequently broken test for now pending a deeper investigation
2020-06-23 14:14:04 +01:00
Kevin Ansfield
db66f1cbbd Fixed populateImageSizes handling of images in subdir setups
no issue

- when using subdirectories, images can be stored in the database both with and without the subdirectory prefix. We weren't taking that into account and so images without the subdirectory were not having the `/content/images/` prefix removed when passed to the storage adapter resulting in the storage adapter not finding the image
2020-06-23 12:47:20 +01:00
Kevin Ansfield
b1924bac1c 🐛 Fixed potentially squashed images in image cards
no issue

- 3.21.0 added `width/height` attributes to `<img>` elements output by image cards which was a potentially breaking change for any sites whose CSS does not expect those attributes, resulting in squashed and distorted images
- updated to `@tryghost/kg-default-cards` which modifies the image card renderer to not output `width/height` attributes
2020-06-23 08:54:58 +01:00
Nazar Gargol
b78dbfbbc2 Removed use of deprecated setting flags from the codebase
refs #10318
refs https://github.com/TryGhost/Ghost/pull/11942

- Removes force_i18n, permalinks, and members_session_secret usage from the codebase
-  We deprecated these flasgs and have not used since Ghost v2. It's good time to remove them before we introduce bigger changes to how `settings` table opeartes.
- Fixed importer test. The test was meant to check if string values were converted properly, the check agains boolean didn't make much sense in this context, so removed it.
- Following this change are going to come ralated migrations to fix existing data (see ref)
2020-06-23 11:58:19 +12:00
Kevin Ansfield
edd997402b Added sizes attribute to in-content <img> elements
no issue

- by including `sizes` with media-queries that match typical content widths we can give the browser better hints as to which responsive image size to fetch resulting in more performant pages
- also fixes `srcset` output where "unknown" sizes were used instead of linking to the original image src which was causing 302 redirects and slowing image loading in some cases
2020-06-19 11:41:32 +01:00
Nazar Gargol
93e8ee83d9 Extracted members CSV related code into @tryghost/members-csv package
no issue

- Moves out CSV parsing and serialization related code into separate package as a part of push to modularize Ghost repo.
- Next up is to remove `csv-parser` dependency from this new package
2020-06-19 19:55:47 +12:00
Nazar Gargol
fabe06c5c5 Colocated readCSV util with members input serializer
refs 5a6ce5abfe

- Allows to move files and keep the history in one go.
- 'csv-parser' will be upgraded to 'papaparse' lib in a new package
2020-06-19 18:24:51 +12:00
Kevin Ansfield
d5caba10d9 Fixed errant .only in tests 2020-06-18 15:35:40 +01:00
Kevin Ansfield
2553da1bee Fixed mobiledoc.populateImageSizes handling of empty image card payloads
no issue

- it's possible that image cards can be saved when they have no `src` attribute (placeholders)
2020-06-18 15:32:42 +01:00
Kevin Ansfield
4fb39f29cc Fixed population of image sizes when forcing re-render of posts
no issue

- fixed incorrect method name when calling
- fixed problem with setting post.mobiledoc to a promise rather than waiting for the size population to finish and setting it to a mobiledoc string
2020-06-18 14:02:53 +01:00
Kevin Ansfield
f27282bc3f
Added mobiledoc transform to populate missing image sizes on forced re-render (#11924)
no issue

- adds `populateImageSizes()` to our mobiledoc lib module
  - uses `image-size` lib to speed up reading of image dimensions
  - for local images, use storage adapter with same guards as used by `handle-image-sizes` middleware so that we don't insert srcsets for images that aren't transformable
  - for unsplash images, remove any width and crop params from the url so it points to the full-size image
- use `populateImageSizes(mobiledoc)` to modify post model's mobiledoc when re-rendering
2020-06-17 13:12:32 +01:00
Kevin Ansfield
32dd4574af Fixed non-reverted rewire of imageSize.getImageSizeFromUrl in tests
no issue

- fixes unexpected behaviour in tests run after the `cached-image-size-from-url_spec.js` unit tests
2020-06-16 20:58:02 +01:00
Hannah Wolfe
a9759736d6
🐛 Fixed private blogging exposing 404 and robots (#11922)
- There were various cases where it was possible to trigger a private site to display a 404 instead of redirecting to /private/
- Private mode was also not always displaying the correct robots.txt
- This PR includes tests for all cases in test/frontend-acceptance/default_routes_spec.js & where possible the unit tests have also been updated for completeness
- Fixing the 404 issues required
    - Better handling of paths using req.path instead of req.url in filterPrivateRoutes
    - Additional error handling, to cover the case that a tag/author RSS feed does not exist
 - Fixing the robots.txt required the order of middleware to be changed, so that private blogging gets a chance to render first
    - NOTE private blogging is the only app with a setupMiddleware function so nothing else is affected
2020-06-16 11:42:32 +01:00
Fabien O'Carroll
8a74cd9e11 Fixed Redirects API regression tests
no-issue

These tests were not testing the functionality correctly, but trying to
test it based on metadata, which can become out of date. This tests the
expected responses and is resilient for future changes to fixtures.
2020-06-16 10:24:13 +02:00
Fabien O'Carroll
c5b8dab523 🐛 Fixed custom redirects for subdirectory setups
no-issue

This issue only occurs when using custom redirects with a subdirectory
setup, and the path to be redirected from is expressed as a regex, and
the url that is being redirected to is not an external url.

The issue has a few components:

- Redirect paths as a regex generally use the ^ to ensure that they
  match the beginning of the path.

- The path that the regex is matched against conditionally excludes the
  subdirectory, specifically, the subdirectory is excluded for external
  urls

These combined means you end up with a regex like /^\/custom-redirect/
and a path like /subdir/custom-redirect, these will not match/replace
correctly, and you'll end in an infinite redirect loop.

The fix here is to *always* remove the subdirectory when testing regex's
and then conditionally adding it back *only* for the redirect, and only
if it is an internal redirect
2020-06-16 10:24:13 +02:00
Nazar Gargol
5f00619d1a Renamed members import/export endpoints
refs cb26fd9305

- Reasoning behind the rename in referenced commit
2020-06-16 18:08:50 +12:00
Hannah Wolfe
3c610a1aff Rework private blogging unit tests
- the existing tests were extremely confusing as tests for logged in and out states were mushed together
- tests for the special private rss feeds only existed for one state, but shouldexist for both
- hopefully it's clearer what's being tested!
2020-06-15 21:04:36 +01:00
Hannah Wolfe
f4641aabe4 Rename private blogging mw functions
- This is just a nicety, trying to make it easier to follow the logic of private blogging
2020-06-15 20:55:59 +01:00
Hannah Wolfe
4ae907781a Private blogging misc cleanup
This is a bunch of small changes, that simplifies working with the private blogging module:
- remove reference to really old paginated RSS behaviour
- remove handling for /rss and allow our standard redirects to redirect to /rss/ and then execute
- readd should to tests so that they can be run standalone
- fix eslint warning
2020-06-15 20:16:13 +01:00
Kevin Ansfield
ce8bc9d4bf Added imageOptimization:srcsets config to allow disabling of srcset attrs in content
no issue

- pass `srcsets` value through to mobiledoc renderer
  - it will stop adding `srcset` attributes to images when explicitly set to `false`
- adds `reload()` method to `mobiledocLib` so that memoized instances of the renderer can be reset during tests that change config values
2020-06-15 16:45:36 +01:00
Kevin Ansfield
73e14b4235 🐛 Fixed small images appearing smaller in post content
closes https://github.com/TryGhost/Ghost/issues/11913

- updated `@tryghost/kg-default-cards` with fixed `srcset` behaviour
  - does not output `srcset` unless we know the image's original size
  - does not output `w` values for sizes that are bigger than the original image
2020-06-15 16:24:02 +01:00
Kevin Ansfield
6535f2ef75 Fixed regression tests
refs 937df2c7ec

- some mobiledoc->html output with blank img alt attributes was missed in regression tests
2020-06-15 09:41:11 +01:00
Kevin Ansfield
937df2c7ec ️ Added blank alt attributes to images when no alt text is provided
no issue

- screen readers often resort to reading out the filename when alt text is not provided which is not useful. By adding `alt=""` (or just `alt`) to `<img>` elements the image is skipped by screen readers providing a smoother experience
- updated `@tryghost/kg-default-cards` which contains above change for `image` and `gallery` cards
2020-06-15 09:24:15 +01:00
Rishabh Garg
84d0a46c6e
Removed "from address" overwrite in importer (#11914)
refs https://github.com/TryGhost/Ghost/issues/11414

- Importing data currently overwrites the existing "from address" with new value
- "from address" needs to go through email validation flow before update which was bypassed
- Updates importer to not allow overwrite for "from address" and use existing
- Adds test for "from address" overwrite
2020-06-15 12:22:55 +05:30
Nazar Gargol
cb26fd9305 Changed members upload validation endpoint to /members/upload/validate
no issue

- The intention is to move away from using file type names in URLs. This endpoint is meant to correspond to current `POST /members/csv` endpoint, that is  planned to be renamed into `/members/upload`. And the `GET /members/csv` to be renamed to  `/members/download` respectively.
2020-06-12 23:31:56 +12:00
Kevin Ansfield
f4d9a41d3b
Added ?paid query parameter to Admin API members browse endpoint (#11892)
no issue

- NQL does not support the relationship setup that members->stripe customer<->stripe subscriptions uses so it wasn't possible to use the `filter` param to query against having an active subscription
- adds `customQuery` bookshelf plugin that allows customisation of SQL query used in `findPage` method by individual models
- use `customQuery` in Member model to set up joins and conditionals to select free/paid members when `options.paid` is present
- allow `?paid` param through API and permitted options for member model
2020-06-12 12:12:10 +01:00
Kevin Ansfield
f1291058ec Fixed regression tests 2020-06-12 10:08:10 +01:00
Nazar Gargol
7904c303a7 Added invalid import record errors and counts
no issue

- This new format allows to return additional metadata with failed import records. The data for invalid records is returned in following format:
```
{
    count: {count_of_invalid_records},
    errors: [{
      message:	"Members not imported. Members with duplicate Stripe customer ids are not allowed." // message field of the error
     context:	"Attempting to import members with duplicate Stripe customer ids." // context field of the error
     help:	"Remove duplicate Stripe customer ids from the import file, and re-run the import." // help field of the error
     count:	2 // count of this specific error
    }]
};
- Errors are grouped by their context fields because message fields sometimes can contain unique information like Stripe customer id, which would produce too many errors in case of bigger datasets.
2020-06-12 19:59:36 +12:00
Nazar Gargol
589d826afd Added /members/validate ednpoint to Admin API
no issue

- This endpoint is meant to be used for validation of imported members
- Main function at the moment is to validate if stripe_customer_id present in the dataset exists in connected Stripe account
2020-06-12 16:34:12 +12:00
Kevin Ansfield
50846e310f Fixed post model regression test
no issue

- `html` output contains `srcset` since 70b205618f
2020-06-11 14:02:53 +01:00
Kevin Ansfield
70b205618f Added responsive-image srcset attribute to image and gallery card output
no issue

- adds a set of hardcoded "content image sizes" to the base config
- adjusts `handle-image-sizes` middleware to always allow the hardcoded content image sizes to be genreated
- updates `@tryghost/kg-card-factory` to allow passthrough of options to card renderers
- updates `@tryghost/kg-default-cards` to add `srcset` output for image and gallery cards
2020-06-11 13:28:44 +01:00
Nazar Gargol
81fc5f8eda Added special handling for member import with Stripe connection
no issue

- When imported member contains stripe_customer_id data but there is no Stripe configured on the Ghost instance such import should faiil. The logic is consistent with one where import fails after not being able to find customer in linked Stripe account
- Fixed import stats to show import failures instead of "duplicate" when the validation error is of "Stripe" origin
2020-06-09 23:02:38 +12:00
Daniel Lockyer
d44dae0c79 Fixed wrong use of moment
no issue

- whilst updating moment, a test that was doing timezone calculation was
  failing. We were only importing moment before, and not moment-timezone
- this commit fixes that
2020-06-08 19:43:55 +01:00
Hannah Wolfe
d541a14826 Change theme uploads to move & delete at end
- Currently theme uploads delete the existing theme before copying the new files into place
- If something goes wrong with the delete action, you will end up in a bad state
   - Some or all of the files may be deleted, but now Ghost won't try to put the new theme in place, instead returning an error
   - This leaves you with an invalid active theme and a broken site
- Unlike delete, move is a one-hit operation that succeeds or fails, there moving a theme is safer than deleting
- This updated code moves the old theme to a folder with the name [theme-name]-[uuid] before copying the new theme into place
- Even if this fails, the files should not be gone
- There's a cleanup operation to remove the theme backup at the end, but we don't care too much if this fails
2020-06-08 16:12:17 +01:00
Kevin Ansfield
db68560b11 🐛 Improved error message output when oembed request fails
refs https://github.com/TryGhost/Ghost/issues/11212

- if a bookmark card fetch is performed (either directly or from fallback) and the page does not have an extractable title, return a more specific error message than "No provider found for supplied URL."
2020-06-08 15:06:00 +01:00
Nazar Gargol
633ba27f0e Added custom label assignment to imported members
no issue

- There is a need to be able to label certain import group of members
with custom labels. This will allow to distinguish/filter these newly
imported  members.
- Allowed `POST /members/csv/` endpoint to accept `labels`
field parameter which assigns labels to every member from imported csv.
2020-06-05 16:22:09 +12:00
Nazar Gargol
12c8b63a4a Added more specific error handling when adding duplicate labels
no issue

- Similarly to other additive api methods  (e.g. members.add) returned more specific ValidationError with contex filled in with the reason why adding did not succed.
- This change is needed for more graceful label handling when adding new members through import
2020-06-05 00:23:10 +12:00
Nazar Gargol
59c773fb04 Removed cache invalidation header when adding label through Amdin API
no issue

- Adding labels doesn't cause any content to invalidate, similarly to adding members. Unlike it's caunterpart - tags, there is no dependent "frontend" content that would become invalid
2020-06-04 23:44:19 +12:00
Fabien O'Carroll
37a1947fc5 Fixed broken regression tests from #11861
no-issue

The config is now a class and the context of `this` was not bound
2020-06-02 16:20:57 +02:00
Kevin Ansfield
64ed246d03
Merge pull request from GHSA-4m2q-w26j-h268
no issue

- added an `externalRequest` lib
  - uses same underlying `got` module as our `request` lib
  - uses `got`'s `beforeRequest` and `beforeRedirect` hooks to perform it's own dns resolution for each url that's encountered and aborts with an error if it resolves to a private IP address block
  - includes a bypass for Ghost's configured url so that requests to it's own hostname+port are not blocked
- updated v2 and canary oembed controllers to use the `externalRequest` lib
2020-06-02 14:30:10 +01:00
Fabien O'Carroll
44f1c0a7b6 Updated members config to respect stripeDirect
no-issue

If the stripeDirect config value is NOT set / false, then connect token
should be used over the API keys if both are present, or else use
whichever is present.  If the stripeDirect config value is set /
true,then API keys should be used and stripe connect token should be
completely ignored.

The reason for this is that we want to use Stripe Connect as the primary
auth method going forward, but we want to keep the direct version
availiable should the Ghost Foundation ever dismantle. This will allow
people to use all the same features with no dependency on an external
service.
2020-06-02 15:28:42 +02:00
Fabien O'Carroll
5b058d6d5b Added permission for auth:members_stripe_connect
no-issue

This permission is used to ensure that only the Owner can
authenticate/authorize the members_stripe_connect integration.
2020-06-02 15:28:42 +02:00
Fabien O'Carroll
95acbbad93 Added stripe_connect_integration default setting
no-issue

This will be used to store the keys and other information after a
sucessful Stripe Connect OAuth flow.
2020-06-02 15:28:42 +02:00
Fabien O'Carroll
eb962d8e5c Added stripe connect module to member service
no-issue

This module handles the creation of a url used for authorization of
Stripe Connect, and also the parsing of the data eventually received
from the authorization flow.
2020-06-02 15:28:42 +02:00
Rish
a7ede2768b 🐛 Fixed test email permissions for non-owner roles
closes https://github.com/TryGhost/Ghost/issues/11841

- Migration adds mapping between permissions and roles for email_preview send test mail
- Only owner previously had correct permission to send test emails
- Fixture existed to allow Admin/Editor/Integrations to send test mails but had missing migration
- Adds tests for roles to send test email
2020-06-02 14:54:50 +05:30
Vikas Potluri
935acfdb88
Updated tests locations and added linting of core/shared (#11872)
* 🔥 removed duplicate error tests

* add lint:shared package script, and updated lint to run it as well

* moved test/unit/config to test/unit/shared/config
2020-06-01 22:06:50 +01:00
Hannah Wolfe
dd9bf73a0b Updated edit URLs to use consistent correct URLs
- Ghost Admin has a redirect system built in, so not using the # does work
- However, the usage of # or no # was inconsistent, which is always wrong
- Using the correct Ghost Admin URLs makes it easier to reason about the behaviour
2020-06-01 08:42:11 +01:00
Aileen Nowak
c779ed0bce 🐛 Fixed iframe script for AMP not injected
closes #11762

- iframe embeds which include a line break were undetected by the regex which checks for the existence and injects the required amp script
- this fix updates the regex to include any non-word character, as line-breaks are not included in the general `.` token
2020-06-01 16:06:13 +12:00
Vikas Potluri
00c324fa4e
Moved core/server/lib/common/logging to core/shared/logging (#11857)
- Represents that logging is shared across all parts of Ghost at present
  * moved core/server/lib/common/logging to core/shared/logging
  * updated logging path for generic imports
  * updated migration and schema imports of logging
  * updated tests and index logging import
  * 🔥 removed logging from common module
  * fixed tests
2020-05-28 19:30:23 +01:00
Vikas Potluri
c7127d5a93
Removed final non-destructured imports of core/server/lib/common (#11858)
* removed final non-destructured imports of core/server/lib/common
* fixed lint
2020-05-28 17:54:18 +01:00
Vikas Potluri
1bd8c18a16
Moved core/server/lib/url-utils to core/shared/url-utils (#11856)
* moved url-utils from server to shared
* updated imports of url-utils
2020-05-28 11:57:02 +01:00
Kevin Ansfield
35f8042d7b
Added ?search= param to Admin API members endpoint (#11854)
no issue

- adds `search` bookshelf plugin that calls out to an optional `searchQuery()` method on individual models to apply model-specific SQL conditions to queries
- updated the base model's `findPage()` method to use the search plugin within `findPage` calls
- added a `searchQuery` method to the `member` model that performs a basic `LIKE %query%` for both `name` and `email` columns
- allowed the `?search=` parameter to pass through in the `options` object for member browse requests
2020-05-28 10:14:02 +01:00
Vikas Potluri
15d9a77092
Moved config from server to shared (#11850)
* moved `server/config` to `shared/config`
* updated config import paths in server to use shared
* updated config import paths in frontend to use shared
* updated config import paths in test to use shared
* updated config import paths in root to use shared
* trigger regression tests
* of course the rebase broke tests
2020-05-27 18:47:53 +01:00
Vikas Potluri
c17a6e457c
Removed import of common sub-modules (#11849)
* replaced import of `common/logging` with destructuring the "module" in migrations
* removed reference to `index` in admin regression test
2020-05-26 21:12:13 +01:00
Vikas Potluri
aeee302c9d
refactored 'lib/image' imports to use destructuring (#11847)
* refactored 'lib/image' imports to use destructuring
* trigger all-tests
2020-05-26 19:11:23 +01:00
Hannah Wolfe
c86933f44f
Remove common errors (#11848)
* refactored core/frontend/services/proxy to import common dependency like a normal person
* removed all imports of `common/errors`
* 🔥 removed common/errors module

Co-authored-by: Vikas Potluri <vikaspotluri123.github@gmail.com>
2020-05-26 19:10:29 +01:00
Kevin Ansfield
071ab9774b
Added Admin API endpoint for basic member stats (#11840)
no issue

- moves members stats generation for the admin graph from the client to the server
- outputs a basic totals count across a requested date range of 30, 90, 365 days, or all time. See below for the response shape
- leaves heavy lifting of the counts to the SQL engines - tested on a dataset of 100k members and query performance is <100ms

```
GET /ghost/api/canary/members/stats/?days=30

{
    total: 100000,
    total_in_range: 20000,
    total_on_date: {
        '2020-04-25': 19000,
        '2020-04-26': 19500,
        // continues until today's date
    },
    new_today: 200
}
```
2020-05-26 10:38:42 +01:00
Vikas Potluri
a9cb8adacc
Refactored tests to destructure common lib import (#11838) 2020-05-25 09:49:38 +01:00
Vikas Potluri
4ac88dce10
Refactored common lib import to use destructuring (#11835)
* refactored `core/frontend/apps` to destructure common imports
* refactored `core/frontend/services/{apps, redirects, routing}` to destructure common imports
* refactored `core/frontend/services/settings` to destructure common imports
* refactored remaining `core/frontend/services` to destructure common imports
* refactored `core/server/adapters` to destructure common imports
* refactored `core/server/data/{db, exporter, schema, validation}` to destructure common imports
* refactored `core/server/data/importer` to destructure common imports
* refactored `core/server/models/{base, plugins, relations}` to destructure common imports
* refactored remaining `core/server/models` to destructure common imports
* refactored `core/server/api/canary/utils/serializers/output` to destructure common imports
* refactored remaining `core/server/api/canary/utils` to destructure common imports
* refactored remaining `core/server/api/canary` to destructure common imports
* refactored `core/server/api/shared` to destructure common imports
* refactored `core/server/api/v2/utils` to destructure common imports
* refactored remaining `core/server/api/v2` to destructure common imports
* refactored `core/frontend/meta` to destructure common imports
* fixed some tests referencing `common.errors` instead of `@tryghost/errors`
   - Not all of them need to be updated; only updating the ones that are
causing failures
* fixed errors import being shadowed by local scope
2020-05-22 19:22:20 +01:00
Kevin Ansfield
21d38b2d37 Added label to known filter params for members
no issue

- updated `filter` plugin with appropriate label relationship and replacement config for NQL
- fleshed out member and label fixture data and tools to facilitate tests
2020-05-21 18:44:52 +01:00
Kevin Ansfield
225544331c Split members admin API tests between acceptance and regression
no issue

- moved typical "happy path" tests for members admin API from regression to acceptance
2020-05-21 18:44:52 +01:00
Fabien O'Carroll
4c3bd0e2e9 Added stripeDirect config to admin api w/ default
no-issue

The flag currently defaults to `true` as we are still using stripe
direct. We expose it on the admin api config endpoint so that the
Ghost-Admin client can use it to conditionally render
2020-05-20 16:47:58 +02:00
Aileen Nowak
86ab62704b
🐛 Fixed structured data issue for publisher logo (#11826)
closes #11304

- Google requires an `ImageObject` to be always returned for `publisher.logo` (https://developers.google.com/search/docs/data-types/article)
- The previous fix 3f5daa60c8 added a second nested `url` error and got therefore reverted with 7ac614030d
- This commit updates the image object generation fn to **always** return an `ImageObject` with minimum of `url` and `@type` properties. If dimensions are available, we'll pass those in there as well
2020-05-20 18:16:20 +12:00
Nazar Gargol
2d41e5cc88 🐛 Fixed permission for "Administrator" to be able to edit post visibility
closes #11825

- The initial implementation had a typo in a role name which didn't allow "Administrator" to edit post's "visibility" attribute
- Added unit tests to check administrator specific role and visibility attribute permission
2020-05-20 17:47:27 +12:00
Hannah Wolfe
2876178dcf 🐛 Fixed logic error in navigation for isSecondary
closes #11772

- Ensures that isSecondary is a boolean true or false
- Added tests that cover the bug, switching to using compile because the helpers have to be run together
- TODO: all tests for helpers should be switched to compile, it's SO MUCH easier
2020-05-18 19:15:28 +01:00
Hannah Wolfe
998eb62e22 Added success indicator for members magic links
- Add a query param that indicates whether signin/up succeeded or failed
- Add unit tests for all 3 possible cases for the createSessionFromMagicLink middleware
- Added an acceptance test to show the behaviour works in principle
2020-05-08 13:17:51 +01:00
Naz
c84866dda7
Improved password reset and session invalidation for "locked" users (#11790)
- Fixed session invalidation for "locked" user
  - Currently Ghost API was returning 404 for users having status set to "locked". This lead the user to be stuck in Ghost-Admin with "Rousource Not Found" error message.
  - By returning 401 for non-"active" users it allows for the Ghost-Admin to redirect the user to "signin" screen where they would be instructed to reset their password

- Fixed error message returned by session API
  - Instead of returning generic 'access' denied message when error happens during `User.check` we want to return more specific error thrown inside of the method, e.g.: 'accountLocked' or 'accountSuspended'
  - Fixed messaging for 'accountLocked' i18n, which not corresponds to the
actual UI available to the end user

- Added automatic password reset email to locked users on sign-in
  - uses alternative email for required password reset so it's clear that this is a security related reset and not a user-requested reset

- Backported the auto sending of required password reset email to v2 sign-in route
  - used by 3rd party clients where the email is necessary for users to know why login is failing

Co-authored-by: Kevin Ansfield <kevin@lookingsideways.co.uk>
2020-05-05 19:37:53 +01:00
Rishabh Garg
a01bcdd2d0
Added new endpoint for refreshing api key secret (#11791)
no issue

- Adds new endpoint on integration to refresh admin/content api key secret
- Allows owner/admin to refresh their content or admin API keys for an integration via Ghost Admin
- Adds a new `refreshed` event to actions table for anytime an api_key secret is refreshed
2020-05-05 23:36:21 +05:30
Hannah Wolfe
53d14fd8e3 Added Router etc to shared/express + use everywhere
- Added a wrapper around express.Router to our shared/express util
- Also export static and _express
- Use this shared util everywhre, meaning express is only used directly in this one file
- ATM this file is mostly an experiment / debug helper, it might be removed again later
- The aim is to have a minimal framework wrapping express that allows us to:
     - reduce our usage of express() in favour of Router()
     - unify some of our duplicated logic
     - fix some structural issues e.g. Sentry
     - make it easier to understand the codebase
2020-05-01 19:32:57 +01:00
Hannah Wolfe
c70c49258e Added new members/api/site endpoint
- easy way to access public settings needed for building members clients
- no auth means this is for public info only
2020-04-30 19:50:40 +01:00
Hannah Wolfe
0cba02b370 Renamed members ssr + api endpoints 2020-04-30 19:00:37 +01:00
Hannah Wolfe
985bbbd194 Moved members routing+mw into its own app
- create a new app for the /members/ endpoint
- moved all /members/ routes and middleware onto this app
- helps to separate members and frontend/site logic so we can start to decouple things more
2020-04-30 13:03:27 +01:00
Hannah Wolfe
3133c302fc Fixed error in redirect regression test
- this redirect is the only one in our tests which is permanent
- also I verified that this new test works the same before and after the recent changes :)
2020-04-30 09:01:49 +01:00
Federico Tibaldo
4fcc31015b
🐛 Fixed regex match replacement when dealing with external URLs (#11781)
refs #10898

- Execute string replacement on external paths
- Take non-top-level base URLs into consideration (to avoid #10776 dups)
- Added tests for all of the above cases
2020-04-30 07:51:36 +01:00
Hannah Wolfe
22e13acd65 Updated var declarations to const/let and no lists
- All var declarations are now const or let as per ES6
- All comma-separated lists / chained declarations are now one declaration per line
- This is for clarity/readability but also made running the var-to-const/let switch smoother
- ESLint rules updated to match

How this was done:

- npm install -g jscodeshift
- git clone https://github.com/cpojer/js-codemod.git
- git clone git@github.com:TryGhost/Ghost.git shallow-ghost
- cd shallow-ghost
- jscodeshift -t ../js-codemod/transforms/unchain-variables.js . -v=2
- jscodeshift -t ../js-codemod/transforms/no-vars.js . -v=2
- yarn
- yarn test
- yarn lint / fix various lint errors (almost all indent) by opening files and saving in vscode
- grunt test-regression
- sorted!
2020-04-29 16:51:13 +01:00
Hannah Wolfe
0fe0e09d62 Moved express init + sentry to a shared util
- added core/shared to watched folders in grunt
- moved sentry to shared
- moved express initialisation to a shared file
- always set trust proxy + sentry error handler
- use this new express init everywhere, and remove duplicate trust proxy and sentry error handler code
2020-04-27 18:17:50 +01:00
Hannah Wolfe
18bd10308b Updated misc files to use ES6 variables
- updated various files I noticed were outdated on my travels around the codebase
- doesn't make any more advanced ES6 changes, this is mostly in the persuit of getting rid of var x = y, z = a; lists at the top of files
2020-04-27 16:03:21 +01:00
Hannah Wolfe
4eeb8604dd 🐛 Fixed public/members.js request path
- removed trailing slash typo
- moved tests out of regression & renamed
2020-04-27 12:48:54 +01:00
Hannah Wolfe
a5c8549616 Moved remaining shared mw tests into correct folder
- make sure the test structure follows the main structure for these files
2020-04-23 19:38:12 +01:00
Hannah Wolfe
4a67ea5546
Fixed admin host with port causing infinite redirect (#11767)
closes #11766, refs 7284227f1

- when we changed from host to hostname, more changed than just using the x-forwarded-host if trusted because express req.hostname does not return the port
- this causes issues with an infinite redirect if you try to set a different admin host with a port
- added a test to demonstrate the case, that didn't fail due to an error in the test logic
- switched from redirecting based on req.hostname to using req.vhost.host which has the correct trusted, requested value that we should rely on
- simplified the comparison logic to explicitly compare host with host
2020-04-23 19:00:28 +01:00
Daniel Lockyer
0e4dbc659c Added eslint exceptions in test-suite
no issue

- eslint-plugin-ghost will warn for console.* calls
- we use a mini-hack in the test suite to print out the start time for
  Ghost
- we want to keep these for the mean time, so explicitly allow them for
  now
2020-04-23 09:15:22 +01:00
Hannah Wolfe
d4cd996e20 Use destructuring for more readable redirect code
- This code was a little verbose, which made it hard to see what was happening (it still is a bit)
- Used destructuring to reduce the code
- Renamed a few variables
2020-04-22 20:30:17 +01:00
Hannah Wolfe
19dde146c1 Renamed redirect functions for clarity
- we had urlRedirects, urlRedirects.adminRedirect and adminRedirects
- all do kinda similar things, but for different contexts so for now I've done a minimal renaming for clarity
- and updated some comments!!
- also removed totally unnecessary if res.isAdmin clause, as we don't use that, and it was never true
2020-04-22 19:31:36 +01:00
Hannah Wolfe
37a22edbe9 Refactored cache-control mw to remove dependencies
- cache-control had some logic in it for private blogging + similar logic exists for members in site/app
- having it in 2 places is weird, and having it inside the mw makes the mw less generic/reusable
- instead of requiring config inside the middleware, we pass config in for the one case where this is used
- fixed tests that didn't test anything 🙈
2020-04-22 18:01:01 +01:00
Hannah Wolfe
f4f61b8a3a Moved normalize image mw into api app
- Moved normalize image mw from shared to api as it is not shared (except within the API)
- This file is only used in one part of the app, this updates the code structure to reflect this
- This is one of many similar changes needed to make it easier to refactor to the existing setup
2020-04-22 17:48:54 +01:00
Hannah Wolfe
de9864db30 Moved upload validation mw into api app
- Moved upload validation mw from shared to api as it is not shared (except within the API)
- Co-located the code with the upload middleware, as it's small and gives us a nice API of .upload.single and .upload.validation
- This file is only used in one part of the app, this updates the code structure to reflect this
- This is one of many similar changes needed to make it easier to refactor to the existing setup
2020-04-22 07:37:02 +01:00
Hannah Wolfe
1a4506dcf0 Moved validation utils into validation/upload
- These two functions have no dependencies and are only used in valiation/upload
- Co-locating the code makes it easier to move
- Exported them with a new module.exports._test pattern - we'll see about whether this is a good idea
- This is one of many similar changes needed to make it easier to refactor to the existing setup
2020-04-22 07:21:41 +01:00
Hannah Wolfe
d85c634669 Removed unused middleware code
- removed empty theme-handler_spec.js file
- removed unused export of uncapitalise
- removed unused export of utils
- remove unused blog-icon and profile-image validation code (now lives in core/server/api/[version]/utils/validators/input)
2020-04-22 07:15:51 +01:00
Hannah Wolfe
05cf4f495d Moved update-user-last-seen mw into api app
- Moved update-user-last-seen from shared to api as it is not shared (except within the API)
- This file is only used in one part of the app, this updates the code structure to reflect this
- This is one of many similar changes needed to make it easier to refactor to the existing setup
2020-04-22 07:12:25 +01:00
Hannah Wolfe
92af5b8f09 Moved api cors mw into api app
- Moved api cors from shared to api as it is not shared (except within the API)
- This file is only used in one part of the app, this updates the code structure to reflect this
- This is one of many similar changes needed to make it easier to refactor to the existing setup
2020-04-22 07:12:25 +01:00
Hannah Wolfe
41c3b4e92b Moved version-match mw into api app
- Moved version-match from shared to api as it is not shared (except within the API)
- This file is only used in one part of the app, this updates the code structure to reflect this
- This is one of many similar changes needed to make it easier to refactor to the existing setup
2020-04-22 07:12:25 +01:00
Hannah Wolfe
213f0a4785 Moved ghost-locals mw into parent app
- Moved ghost-locals from shared to parent as it is not shared
- This file is only used in one place, this updates the code structure to reflect this
- This is one of many similar changes needed to make it easier to refactor to the existing setup
- This allows shared middleware to be unhooked from the parent app
2020-04-21 18:50:46 +01:00
Hannah Wolfe
8fe56852e0 Moved request-id mw into parent app
- Moved request-id from shared to parent as it is not shared
- This file is only used in one place, this updates the code structure to reflect this
- This is one of many similar changes needed to make it easier to refactor to the existing setup
2020-04-21 18:29:20 +01:00
Hannah Wolfe
becc239de1 Moved web/parent-app to web/parent/app
- Have a consistent structure so we can co-locate an app with its own middleware
- This is another small step in sorting out the giant mess that was web/shared/middleware
2020-04-21 18:08:29 +01:00
Hannah Wolfe
61633d48ee Moved serve-public-file mw into site app
- Moved serve-public-file from shared to site as it is not shared
- This file is only used in one place, this updates the code structure to reflect this
- This is one of many similar changes needed to make it easier to refactor to the existing setup
2020-04-21 17:23:07 +01:00
Hannah Wolfe
406218489c Remove additional mount of content/images
refs 717567995b

- We should not be using relative URLs inside the admin panel anymore
- Removes a usage of "shared" middleware which isn't truly shared
2020-04-21 17:02:16 +01:00
Hannah Wolfe
dbafaf7205 Moved handle-image-sizes mw into site app
- Moved handle-image-sizes from shared to site as it is not shared
- This file is only used in one place, this updates the code structure to reflect this
- This is one of many similar changes needed to make it easier to refactor to the existing setup
2020-04-21 15:50:01 +01:00
Hannah Wolfe
9fe741c9b9 Moved static-theme mw into site app
- Moved static-theme from shared to site as it is not shared
- This file is only used in one place, this updates the code structure to reflect this
- This is one of many similar changes needed to make it easier to refactor to the existing setup
2020-04-21 15:50:01 +01:00
Hannah Wolfe
40fa2053e3 Moved serve-favicon mw into site app
- Moved serve-favicon from shared to site as it is not shared
- This file is only used in one place, this updates the code structure to reflect this
- This is one of many similar changes needed to make it easier to refactor to the existing setup
2020-04-21 14:30:32 +01:00
Kevin Ansfield
9981ea336c Changed {subscriber_firstname} replacement to {first_name}
no issue

- easier to remember and type
- removes need to remove the `subscriber_` prefix when accessing member properties
2020-04-20 14:25:58 +01:00
Kevin Ansfield
944ffa8668 Added acceptance test for email preview with email card and replacements
no issue

- the email previews API should return html and plaintext content that contains any email cards and where any valid replacement strings have been replaced with desired fallbacks
- unknown/invalid replacement strings should appear as entered by the user (no %% wrappers that are used internally)
2020-04-20 12:48:00 +01:00
Daniel Lockyer
7ac614030d 🐛 Fixed structured metadata for images
fixes #11746

- the original fix[1] inadvertently moved the logo object down one level
- this commit moves it back up, and fixes the tests to reflect the
  expected format

[1]: 3f5daa60c8
2020-04-17 13:51:55 +01:00
Daniel Lockyer
8853067830 Added custom json-string format to AJV validator
no issue

- the value of `mobiledoc` when submitting a page/post via the API must
  be JSON, but we don't validate this
- this results in url-utils throwing an error, which ends up being a 500
- this commit adds a custom format to AJV to validate it is valid JSON
- also updates tests with bad JSON - 'a'
2020-04-17 11:56:20 +01:00
Kevin Ansfield
374b43ceab Fixed linting error 2020-04-17 11:32:29 +01:00
Nazar Gargol
192cb107a5 Fixed regression test for tag edit redirects
refs bd92fcd282

- Referenced commit broke the regression test
2020-04-16 22:10:28 +12:00
Hannah Wolfe
a4cf470c87 🐛 Fixed importer always erroring
- in 3.13.2 the importer always throws the error "The "path" argument must be of type string. Received an instance of Object"
- this is due to a change in method signature that wasn't accounted for
- added a test to catch similar changes to this code in future
2020-04-15 13:24:43 +01:00
Hannah Wolfe
fc664ac697 Improved helper index, use glob to read directory
- Having to remember to add files to an index.js is a PITA
- We already have glob as a dependency, so use that
- This requires that the file prev_next be renamed to prev_post, which is what it's called as by default
- next_post is an alias of next_post - so this reflects that more closely
2020-04-10 13:08:41 +01:00
Hannah Wolfe
59b9f161dd Moved non-helper code out of helpers
- the helper dir also contained some code used with helpers - utils and helper-helpers?
- the goal here was for helpers to be the only thing in their folder so we can look at moving them out
- all other code has been moved to services/themes for now, which is not the right place either
- services/themes is a catch-all for theme storage, loading, validation, rendering and more, needs to be broken down
2020-04-10 12:27:43 +01:00
Hannah Wolfe
3a2ccea3a9 Moved helper util tests to Ghost-SDK/helpers
ref 99b2adc1c82eface95dca2a85d6f62169265d3e4

- these tests belong to @tryghost/helpers as they test code that isn't in this repo
2020-04-10 08:19:16 +01:00
Hannah Wolfe
e639a25171 Moved labs usage into cancel_link helper file
- this removes the need to require the proxy inside the helpers/index.js file
- have the helper file define and return it's own labs-enabled-ness
- update the tests to test for the labs flag being unset
2020-04-08 19:31:55 +01:00
Kevin Ansfield
b37ac8ef1f Switched to extracted @tryghost/mobiledoc-dom-renderer package
no issue

- moved `mobiledoc.renderers.mobiledocHtmlRenderer` to `mobiledoc.mobiledocHtmlRenderer` so that it's easier for the getter to access the parent objects getters
- removed all tests and dependencies that now live in @tryghost/mobiledoc-dom-renderer
  - kept the `mobiledocHtmlRenderer` test because that's testing that we've correctly wired up our cards and atoms and the output is what we expect
2020-04-08 18:27:12 +01:00
Hannah Wolfe
35e3e0708c Moved helper proxy into a service
- The proxy is not a helper, we want the helpers folder to only include helpers
- The proxy is also meant to be the interface to Ghost for the helpers, and we want to enforce that
- This is a small step on the way
2020-04-08 17:22:44 +01:00
Kevin Ansfield
294187c41f Switched to extracted @tryghost/kg-default-atoms package
no issue

- removed code that now lives in the external Koenig monorepo
2020-04-08 14:49:44 +01:00
Tim Gates
270af89ee2
Fixed typo tempalte -> template (#11722)
no issue

- Fixes al typo in test/regression/site/site_spec.js. Should read `template` rather than `tempalte`.
2020-04-08 14:26:50 +12:00
Kevin Ansfield
2642405595
Merge pull request from GHSA-q6jp-j3gg-3mxr
no issue

- backports fixes implemented in https://github.com/TryGhost/Ghost/commit/477393967 from the v3/api endpoint to the v2 endpoint
2020-04-07 11:05:48 +01:00
Joseph Coffland
67b8fbf6cf Added raw handlebars helper
- Allows using the 4-bracket raw block syntax e.g: {{{{raw}}}}{{{{/raw}}}}
- This allows you to include handlebars inside a template that is not compiled and executed
- The common usecase is if you want to include client-side handlebars templates inside server-side ones
2020-04-06 17:26:50 +01:00
Fabien O'Carroll
a701ee7023
Added support for token session to /ghost (#11709)
no-issue

* Added default for getting origin of request

This function is used to attach the origin of the request to the
session, and later check that requests using the session are coming from
the same origin. This protects us against CSRF attacks as requests in
the browser MUST originate from the same origin on which the user
logged in.

Previously, when we could not determine the origin we would return
null, as a "safety" net.

This updates the function to use a secure and sensible default - which
is the origin of the Ghost-Admin application, and if that's not set -
the origin of the Ghost application.

This will make dealing with magic links simpler as you can not always
guaruntee the existence of these headers when visiting via a hyperlink

* Removed init fns and getters from session service

This simplifies the code here, making it easier to read and maintain

* Moved express-session initialisation to own file

This is complex enough that it deserves its own module

* Added createSessionFromToken to session service

* Wired up the createSessionFromToken middleware
2020-04-06 11:49:14 +02:00
Fabien O'Carroll
1f68d8dc20 Refactored existing adapters to use adapter-manager
no-issue

This removes duplications and reduces maintentence going forward.
2020-04-05 21:13:47 +02:00
Fabien O'Carroll
23154f0739
Refactored session service (#11701)
* Refactored SessionStore to use @tryghost/errors

no-issue

* Updated tests to test exposed API

no-issue

This will make refactoring easier, as we only have the "public" contract to maintain

* Refactored session functionality to SessionService

no-issue

This splits the session logic away from the HTTP responding logic,
which will allows us to decouple session creation/modification from the
API. Eventually this can be used to create sessions based on magiclink
style tokens.

* Instantiated and exported the new SessionService

no-issue

* Refactored session middleware to take session service

no-issue

This removes duplication of code and makes the middleware more explicit
that it's just a wrapper around the session service.

* Updated to use external @tryghost/session-service

no-issue
2020-04-02 16:27:31 +02:00
Hannah Wolfe
573850ddd7 Removed fixture yarn.lock file
- This theme fixture is confusing GitHub's security tooling into thinking we have a tonne of insecure dependencies that we don't
2020-03-31 12:56:15 +01:00
Hannah Wolfe
7f1d3ebc07
Move tests from core to root (#11700)
- move all test files from core/test to test/
- updated all imports and other references
- all code inside of core/ is then application code
- tests are correctly at the root level
- consistent with other repos/projects

Co-authored-by: Kevin Ansfield <kevin@lookingsideways.co.uk>
2020-03-30 16:26:47 +01:00