Commit Graph

110 Commits

Author SHA1 Message Date
Naz
58b3c47781 Fixed broken notifications/setting acceptance tests
refs d9ddc2db6a
refs https://github.com/TryGhost/Team/issues/754

- The tests were written with falsy assumptions and validation added in refed commit have uncovered it!
-  A secondary issue touched here is additional JSON object serialization that is used in the  "input serializer" -d9ddc2db6a/core/server/api/v2/utils/serializers/input/settings.js (L107-L110)
- The additional stringification should not be there at all. It covers for a mistaken internal use of Settings API where raw objects are passed around instead of serialized JSON Objects (see commets left with this changeset for details)
2021-06-28 16:48:00 +04:00
Kevin Ansfield
1bc57b584a
Added posts_meta.feature_image_{alt,caption} columns (#13030)
refs https://github.com/TryGhost/Team/issues/770

We want post feature image functionality to better match what's available inside the editor, to do that we'll need somewhere to store alt and caption meta data. `posts_meta` chosen because even though we want to make this generic for other tables in the future those tables also have a `feature_image` (or closely related) field.

- updated schema with new columns
- added migration to create columns
- cleaned new columns from API output
  - not output on v2/v3
  - conditionally output on v4/canary output based on labs flag
- bumped `@tryghost/admin-api-schema` to allow new columns through in canary API requests
  - silently clean properties from input when labs flag is disabled
  - updated acceptance tests so they fail if `admin-api-schema` is not letting the new fields through
2021-06-10 20:35:56 +01:00
Naz
8ab43b84d5 Enabled 'labs' to be accepted as a value in Settings API
refs https://github.com/TryGhost/Team/issues/757

- As labs has been added back to the settings table the APIs are reverting back to accepting it as a value
2021-06-08 02:18:17 +12:00
Fabien 'egg' O'Carroll
2a81d0a986
🐛 Fixed saving Members with Complimentary plans (#13008)
* 🐛 Fixed saving Members with Complimentary plans

refs https://github.com/TryGhost/Team/issues/758

Since 4.6 The Admin is using the comped flag again, rather than creating
subscriptions for zero-amount prices directly. With the `comped` flag
removed, the default state was for it to be falsy in the Admin, and when
saved would trigger the legacy comped flow, cancelling the subscription.

This reverts commit 57a176ff3d.
2021-06-03 18:28:14 +01:00
Hannah Wolfe
0f42bbeac3
Moved acceptance test utils into own file
- starting and stopping Ghost is part of our rather unloved acceptance test framework
- moving them into their own file to make the different pieces clearer and also to start to make improvements
- first improvement had to happen as an aside - exposing the existingData property via a function and making the API clearer
- this was a weird thing set on module.exports, very hidden and hard to follow
- Note: stopGhost is only used once in the regression/modles/model_posts_spec.js file to make the test run fast enough...
- These tests are effectively "DB integration" tests e.g. non-unit tests because they do use the DB, they need their own framework
2021-06-01 13:17:45 +01:00
Fabien O'Carroll
5880edd722 Replaced members 'comped' status with 'paid'
refs https://github.com/TryGhost/Team/issues/693

Since we've got rid of the concept of Complimentary with the Custom
Prices work, we're removing the 'comped' status from members. This
involves a migration for existing members, a schema update for the
validation, and a bump to members-api to no longer use the 'comped'
status for new members.

We also update the aggregation of the MemberStatusEvent to consider the
'comped' status as 'paid', and that there are 0 'comped' status events
in the database.

We can consider a migration for this data in the future, either adding
new status events moving from 'comped' to 'paid', or by modifying
existing status events. However both of these are very difficulty to
write a down migration for, and might be best saved for a major version.

- @tryghost/members-api@1.7.0 is the version that includes the required
  changes, however we have already bumped to 1.8.0 in Ghost
2021-05-19 18:49:18 +01:00
Fabien O'Carroll
57a176ff3d Removed comped flag from canary Members Admin API
refs https://github.com/TryGhost/Team/issues/693

Since we no longer have a concept of "comped" we're removing the flag
from the unstable canary api.
2021-05-19 18:49:18 +01:00
Naz
255b4da7a2 Removed unused test utils modules
no issue

- This is dead code!
2021-05-13 15:41:27 +04:00
Naz
d98f76b18c Refactored test utils "post" properties
refs 06dd9bac59
refs https://github.com/TryGhost/Team/issues/687

- This is a continuation of the work started in the refed commit. In short: allowlisting response checks wherever possible
2021-05-13 15:41:27 +04:00
Naz
06dd9bac59 Refactored post resource Admin API test utils
refs https://github.com/TryGhost/Team/issues/687

- The approach of generating validation properties using `/server/data/schema` package's tables object is prone to leaking unwanted database fields into API responses
- This refactor takes a tiny step into direction of relying on "allowlist" approach for properties in the API response resources.
- Apart from solving the described property leak problem it also moves toward decoupling tests from `/core/server` dependencies!
2021-05-12 18:15:54 +04:00
Kevin Ansfield
c36e749820
Added support for gating content by member labels and products (#12946)
refs https://github.com/TryGhost/Team/issues/581
closes https://github.com/TryGhost/Team/issues/582

Emails can now be sent to members with specific associated labels or products by specifying an NQL string. We want to bring the same members segment feature to content by allowing `visibility` to be an NQL filter string on top of the `public/members/paid` special-case strings.

As an example it's possible to set `posts.visibility` to `label:vip` to make a post available only to those members with the `vip` label.

- removed enum validations for `visibility` so it now accepts any string or `null`
    - bumped `@tryghost/admin-api-schema` for API-level validation changes
- added nql validation to API input validators by running the visibility query against the members model
- added transform of NQL to special-case visibility values when saving post model
    - ensures there's a single way of representing "members" and "paid" where NQL gives multiple ways of representing the same segment
    - useful for keeping theme-level checks such as `{{#has visibility="paid"}}` working as expected
- updated content-gating to parse nql from post's visibility and use it to query the currently logged in member to see if there's a match
    - bumped @tryghost/members-api to include label and product data when loading member
2021-05-10 19:32:11 +01:00
Hannah Wolfe
829e8ed010 Expanded requires of lib/common i18n and events
- Having these as destructured from the same package is hindering refactoring now
- Events should really only ever be used server-side
- i18n should be a shared module for now so it can be used everywhere until we figure out something better
- Having them seperate also allows us to lint them properly
2021-05-03 17:14:52 +01:00
Naz
bc15f8c1bb Added "emails" limit check when (re)sending a newsletter
refs https://github.com/TryGhost/Team/issues/588

- This check allows for a on/off switch to be set up on the instance and control limits around sending emails
- An example configuration for such check would look like following in config's hostSettings section, e.g.:
```
"emails": {
    "disabled": true,
    "error": "Email sending has been temporarily disabled whilst your account is under review."
}
```
2021-05-03 17:51:33 +04:00
Fabien 'egg' O'Carroll
33f26fbf32
Updated subscriptions for Members Admin API
refs https://github.com/TryGhost/Team/issues/616

We need a way to assign Products to Members via a Subscription, and we've
followed the same pattern as the editSubscription method for the Members API
controller, which acts upon Subscriptions as a nested resource.

Subscriptions now are linked to products, and we've included those links by
default in the Member Admin API as we already include subscriptions by
default, and Products are now a core part of the Members feature-set.
2021-04-26 17:14:34 +01:00
Daniel Lockyer
e9b21fdbd1 Updated bson-objectid calls to match API change
refs c873899e49

- as of `bson-objectid` v2.0.0, this library exports the function
  to generate an ObjectID directly, and then you need to use `.toHexString()`
  to get the 24 character hex string - 6696f27d82
- this commit removes all uses of `.generate()` and replaces with this
  change
2021-04-21 16:23:52 +01:00
Naz
cb0807d07a Blocked requests from integrations when integration limit is in place
https://github.com/TryGhost/Team/issues/599

- When custom integration limit is enabled all requests from existing integrations should not be accepted. With the exception of internal integrations like backup and scheduler
2021-04-10 00:45:26 +12:00
Kevin Ansfield
bb19eddeae
Added DELETE /members/ to the Admin API for bulk member deletion (#12082)
refs https://github.com/TryGhost/Team/issues/585

- adds `DELETE /members/` route to the Admin API
- supports `?filter`, and `?search` query params to limit the members that are deleted
- `?all=true` is required if no other filter or query is provided
- uses `models.Member.bulkDestroy` which _will not_ cancel any Stripe subscriptions if members have them but _will_ clean up the Stripe relationship data in Ghost's database
2021-04-08 12:03:45 +01:00
Rishabh Garg
72e8894eac
Cleaned old members stats endpoint (#12821)
no refs

- Removes old `/members/stats` endpoint in favor of new `/members/stats/count` in canary/v4 which captures members counts using new events table
- Removes tests for old `/members/stats` endpoint
- Added test for new `/members/stats/count` endpoint
2021-03-29 13:07:01 +05:30
naz
7f50083cc5
Fixed node version specific object key sorting (#12817)
refs 74cdbadedb

- Fixes failing test by enforcing ordering through `sort()` call. Doing this to avoid differences between node versions' `Object.keys()` ordering
2021-03-25 15:16:24 +13:00
Naz
74cdbadedb Fixed exporter tests using should.have.keys assertion
refs https://github.com/TryGhost/Team/issues/555
refs 080a8fc082

-  The `have.keys` assertion was not doing strict comparison neither provided any useful output when changed to `have.only.keys`.
- Rewrote the tests to use manual assertion through array comparison which checks exactly what it's supposed to and gives a visual diff in case there are any missing/extra properties in config
2021-03-25 13:22:34 +13:00
Naz
32e1ea3679 Moved importer tests to regressions suite
refs 3240d4adf0

- Acceptance tests have low timeout limit and were not making it through with updated export files
2021-03-24 18:11:34 +13:00
Naz
3240d4adf0 Updated export fixture for Ghost 3.0
refs https://github.com/TryGhost/Team/issues/555

- This test fixure is up to date and contains real export from Ghost 3.0. Previous file was just an old 2.x export file which was updated manually on "as needed bases"
- The aim is to keep these files as close to real world as possible instead of manually making up data for the needs of test suites
- Fixed test suite which was using the v3 export file
2021-03-24 15:44:56 +13:00
Thibaut Patel
44035fd591 Added a better error when a user try to create an orphaned webhook
issue https://github.com/TryGhost/Team/issues/526
refs 70627d84a7 (r48575450)
2021-03-23 17:15:21 +01:00
Naz
d30ecf70c0 Added latest export body generator
refs https://github.com/TryGhost/Team/issues/555

- Export/Import test suite clean up ctd. See previous commits for context
- Main goal here was to update latest JSON export data. Also hardened tests to make sure this fixture is updated whenever the endpoint changes structure
2021-03-23 00:42:22 +13:00
Fabien 'egg' O'Carroll
d19452147d
🐛 Fixed @price template data to work with price helper (#12764)
closes https://github.com/TryGhost/Team/issues/545

The price helper requires an object with amount & currency properties to
work correctly. This updates the @price data object to expose these.

In order to maintain backward compatibility with using the @price data
as primitive number values, we add a valueOf method which returns the
legacy dollar amount value.

This means you can use {{price @price.monthly}} OR
{{@price.monthly}} - the second of which will output the dollar
amount.

A new theme fixture was added to test both usages of the @price data
2021-03-18 16:13:10 +00:00
Kevin Ansfield
19d5448101
🐛 Fixed complimentary members' content gating (#12761)
no issue

Comped members were not able to view paid-member content because content gating was only looking for `member.status === 'paid'` which doesn't take into consideration members on a "complimentary" plan.

- added front-end acceptance tests for member access to posts
- updated content-gating check to take comped members into consideration
2021-03-15 19:13:48 +13:00
Hannah Wolfe
fdc043227a Updated fixtures for Ghost v4.0
- With 4.0 we have a brand new version of Casper, new fixtures and new default settings
- Fixture posts cover the key features and give users an introduction to how to use their site
- This all comes from the marketing and design teams to refresh the look and feel of Ghost and give users the best possible onboarding experience
Note: this fixture overhaul includes
- new content for new 4.0 features
- regenerated post content using our updated mobiledoc structure
- a switch from British to US English
2021-03-10 17:37:16 +00:00
Kevin Ansfield
021cfecb59
Fixed __GHOST_URL__ appearing in settings values after edit (#12738)
refs https://github.com/TryGhost/Ghost/pull/12736
refs https://github.com/TryGhost/Team/issues/467

knex's `parse()` method is only called on data when directly fetched from the db. This was causing problems when model instances are passed around via events for example because `.get('key')` will return data that was directly set on the model without having gone through the `parse()` transformations. The result of this inconsistency was settings appearing correct when Ghost started up but then being broken as soon as a setting was changed.

- moved absolute/relative->transform-ready URL transformations from the API input serializers to the model's `format()` method and replaced with a relative->absolute transform in API input serializers
    - results in consistency because `.get()` on a settings model will always return an URL
- removed transform-ready->absolute transforms from the API output serializers as that is now handled at the model-layer
2021-03-08 18:41:43 +00:00
Fabien O'Carroll
f03e1dd863 Excluded member events tables from site exports
refs https://github.com/TryGhost/Team/issues/525

We expect the member event tables to be large, and they contain what is
considered metadata. For this reason we do not want to include them in
the export of sites.
2021-03-05 11:41:00 +00:00
Naz
e7e5f0ad15 Updated API versions used in regression test for webhooks
refs https://github.com/TryGhost/Team/issues/513

- Tests confused versions after doing a "tripplication"
- Extracted a global "API_VERSION" variable for each suite to make api version used in the test explicit
2021-03-04 13:06:16 +13:00
Naz
492f115850 Bumped gscan to 4.0.0-pre.1
refs https://github.com/TryGhost/Team/issues/492
d0e0760dae

- With Ghost 4.0 it will detect deprecated {{@labs.members}}  warning and use of deprecated ghost-api v2 in theme's package.json
2021-03-01 19:26:17 +13:00
Kevin Ansfield
7cec1ecb57 Added acceptance test for github theme installs
refs https://github.com/TryGhost/Team/issues/480

- testing happy path
  - mock github api request with redirect to zip-file download endpoint
  - theme is installed with a name of the github repo
  - response matches a theme upload response
2021-02-24 10:16:25 +00:00
Fabien O'Carroll
7785fb6dc5 Refactored usage of pre v4 "unsplash" setting format
refs https://github.com/TryGhost/Ghost/issues/10318

- API changes introduced:

canary/v4 Admin API
GET /settings/ (browse)

+ "unsplash" present in response as boolean value

GET /settings/:settingName (read)

+ "unsplash" present in response as boolean value

PUT /settings/ (edit)

+ "unsplash" updates setting, accepts ONLY  boolean format

v3  Admin API
GET /settings/ (browse)

+ "unsplash" present in response with object value

GET /settings/:settingName (read)

+ "unsplash" present in response with object value

PUT /settings/ (edit)

+ "unsplash" updates setting, accepts either boolean or object formats

v2 Admin API
GET /settings/ (browse)

+ "unsplash" present in response with object value

GET /settings/:settingName (read)

+ "unsplash" present in response with object value

PUT /settings/ (edit)

+ "unsplash" updates setting, accepts object format
2021-02-18 10:55:59 +13:00
Naz
37ef40b46e Dropped support for labs setting parameter
refs https://github.com/TryGhost/Ghost/issues/10318

- `labs` setting is dropped from setting values as the use of JSON objec
to sore settings has been deprecated
- `labs` setting is no longer accepted as a paramter in the Settings API nor the
impoprter. The value is ignored if present in the POST/PUT requests and
returns 404 in case it is requested by key at `GET /settings/:key`
2021-02-17 19:55:16 +13:00
Naz
ea6d656457 🏗 Removed labs from setting
refs https://github.com/TryGhost/Team/issues/332

- The last value that has been used in the code was "members"
- By default members will be always "on" starting Ghost 4.0, so there's no need for this flag anymore
- Therefore there's no real need to keep "labs" around
2021-02-17 19:55:16 +13:00
Fabien O'Carroll
8880cc82c7 Refactored usage of the pre v4 "slack" setting
refs https://github.com/TryGhost/Ghost/issues/10318

- Change/compatibility table for this changeset:

Importer

+ "slack_username"  import
- "slack_url" ignore
+ "slack.username" import
- "slack.url" ignore

v3 & canary/v4 Admin APIs
GET /settings/ (browse)

+ "slack_username" present in response
+ "slack_url" present in response
+ "slack" present in response

GET /settings/:settingName (read)

+ "slack_username" present in response
+ "slack_url" present in response
+ "slack" present in response

PUT /settings/ (edit)

+ "slack_username" updates setting
+ "slack_url" updates setting
+ "slack" !NOTE! updates setting unless "slack_username" or "slack_ur"l are set (updated keys take priority)

v2 Admin API
GET /settings/ (browse)

- "slack_username" NOT present in response
- "slack_url" NOT present in response
+ "slack" present in response

GET /settings/:settingName (read)

- "slack_username" ignored (404)
- "slack_url" ignored (404)
- "slack" present in response

PUT /settings/ (edit)

- "slack_username" ignored (no error)
- "slack_url" ignored (no error)
+ "slack" updates setting
2021-02-15 15:25:54 +13:00
Fabien 'egg' O'Carroll
601d135b43
Added members event tables (#12618)
refs https://github.com/TryGhost/Ghost/issues/12602

* Added members_payment_events table

This table will store successful and unsuccessful payment attempts, and
can be used to calculate gross volume over time.

* Added members_login_events table

This table can be used to audit member logins

* Added members_email_change_events table

This table will allow us to store a history of email addresses associated with a member

* Added members_status_events table

This table will allow us to track the change in status over time for members, as well
as calculate aggregates over time, e.g. paid members over time

* Added members_paid_subscription_events

This table will allow us to track subscriptions changes for members, as well as 
calculating MRR over time
2021-02-11 10:54:30 +00:00
Naz
e0178099a2 🏗 Altered slug unique constraint in posts table
refs https://github.com/TryGhost/Ghost/issues/12567

- Changing unique constraint from slug to slug+type should allow for posts and pages to be created with the same slug
- The constraint will be present on application layer for API v4 while we figure out how to deal with it in API v5
2021-02-09 13:31:12 +13:00
Fabien O'Carroll
e39644bd2d Added members_subscribe_events table
refs https://github.com/TryGhost/Ghost/issues/12602
2021-02-05 16:02:08 +00:00
Naz
606212d187 🏗 Dropped apps related tables
refs https://github.com/TryGhost/Ghost/issues/12565

- Dropped apps_* tables which were not used since the concept of "apps" has been abandoned
- Uses newly introduced "dropTables" migration utility
- Review for these changes has been completed in
(https://github.com/TryGhost/Ghost/pull/12592)
2021-02-03 12:20:08 +13:00
Rishabh Garg
26ee648397
🏗 Flattened members subscriptions data in API (#12581)
no refs

- Updates member model serializer to directly set subscriptions on member object instead of `stripe.subscriptions`
- Updates all references to members subscriptions from nested `stripe.subscriptions` to `subscriptions`
- Updates v3 API serializer to still use `stripe.subscriptions`
- Updates tests
2021-02-02 16:12:53 +00:00
Fabien 'egg' O'Carroll
6af2706f10
Updated Admin API and Mega to use status flag (#12579)
no-issue

* Removed support for paid param from v3 & canary API
* Updated active subscription checks to use status flag
* Updated MEGA to use status filter over paid flag
* Removed support for paid option at model level
* Installed @tryghost/members-api@1.0.0-rc.0
* Updated members fixtures
2021-02-02 16:12:51 +00:00
Fabien 'egg' O'Carroll
a79ed1170d
Added status property to members (#12570)
refs #12160

This flag will allow us easier filtering of members via the API

* Added status column to members table

This flag will be used to determine if a member is free or paid, rather
than relying on joins with the customers and subscriptions tables.

* Added migration to populate members.status

As we add the column with a default value of "free" we only need to care
about the paid members here. We also preemptively handle migrations for
SQLite where there are > 998 paid members.
2021-02-02 16:11:34 +00:00
Thibaut Patel
b724b2be92
Removed the deprecated send_email_when_published field from the API
refs https://github.com/TryGhost/Team/issues/457
2021-02-02 16:11:34 +00:00
Rishabh Garg
5417c4d0e1 Removed stripe keys and analytics data from export (#12573)
closes https://github.com/TryGhost/Ghost/issues/12449

We’re starting to bump into errors with our current exporter due to the size of some of the tables in the db and hitting an issue with Ghost running out of memory during export. The intention for the export/import is not to be backup/restore functionality, but for exporting content and authors.

In addition, exporting and re-importing Stripe secret/publishable keys can cause unexpected side-effects for sites that can has major side-effects. This change -

- Removes `email_batches` and `email_recipients` tables from export data to reduce export size due to large amount of analytics data
- Removes stripe's secret/publishable/webhook keys to avoid unexpected issues with re-import
2021-01-28 14:55:54 +05:30
Thibaut Patel
01126cf1bb Added acceptance tests on the v3 api version.
refs https://github.com/TryGhost/Team/issues/221
2021-01-22 17:49:06 +01:00
Kevin Ansfield
8aa55feaf8
Added acceptance test for /member/:id/?include=email_recipients (#12477)
refs c1d66f0b01

- fixed base model allowing '@@INDEXES@@' as a permitted attribute/order
- fixed base model automatically setting `@@INDEXES@@` to null on the model when creating
- added `doAuth('members:emails')`
  - creates an `email_batch` record attached to the first email in the fixtures
  - creates an `email_recipients` record for each member
  - runs analytics aggregation so the email and member counts are as expected
- added acceptance test for `/member/:id/?include=email_recipients`
2020-12-11 18:45:35 +00:00
Fabien O'Carroll
32fe260763 Updated canary members controller to use new Importer
no-issue

This completely replaces the old import functionality with the new
importer!
2020-12-09 19:25:57 +00:00
Fabien O'Carroll
73dc19e498 Removed Members CSV import validation endpoint
no-issue

The new import flow does not use prevalidation of the CSV file, so we
have no use for this anymore.
2020-12-09 19:25:57 +00:00
Daniel Lockyer
a7e17c6b98
Changed URL protocol in oembed acceptance tests
- `https` was getting caught somewhere with nock and metascraper and
  caused each test case to hang for 3 seconds
- `http` still tests what we want and is instant
2020-12-03 21:46:01 +00:00