Commit Graph

8507 Commits

Author SHA1 Message Date
Thibaut Patel
9e93ed7ef0 Adding sentry logging to all ping errors
issue TryGhost/Team#362
2021-02-19 14:00:50 +01:00
Hannah Wolfe
b28b4e1cb0 Cleaned up, commented + co-located new boot
- moved the DB init back into the boot file so everything is colocated - want it to be EASY to reason about whats happening
- added some more debug, and made the method signatures all the same
- expanded on comments to make it clear what the intentions are here, as there's a lot more work to do
2021-02-19 12:27:49 +00:00
Daniel Lockyer
db1c1ce21d Changed DB path for SQLite tests
no issue

- previously for SQLite tests we were pointing to a DB file within the
  content/data folder, which would be stored on the filesystem
- by pointing this file to be under `/tmp`, the file is stored in memory
  and should be a lot quicker to access
- this works great for me! - 2x faster test suite for SQLite
- however, Windows doesn't have a `/tmp` folder so we need to rewrite that
  part to replace with `os.tmpdir()`
2021-02-19 12:21:00 +00:00
Hannah Wolfe
71521d166e Readded job scheduling to boot process
- This was missing from the new boot process!
- Added it back because obviously it is necessary :D
2021-02-19 10:09:41 +00:00
Hannah Wolfe
4b472615a8 Added Sentry to new boot process
- With the new boot finalised it's clearer to see where this needs to go
- Update comments to add more clarity around what's happening and why
- Cleaned up the version require, as this was prep for maybe using version via config, which won't work in MigratorConfig
2021-02-19 09:20:41 +00:00
Hannah Wolfe
f1be3418d9 Switched tests to use new boot method
- We currently have 2 boot systems in the codebase, until now the tests were still running against the old one
- Regression tests using the forceStart + subdir flag no longer work. These need reworking, but this _should_ be easier later
- Note those tests work fine if they were the first tests run, it is only the case of trying to restart the server with a subdirectory after starting without that doesn't work
2021-02-18 20:24:49 +00:00
Thibaut Patel
6e58846c94 Added sentry logging to 429 ping errors
issue https://github.com/TryGhost/Team/issues/362
2021-02-18 19:21:15 +01:00
Daniel Lockyer
a2dd7c7f7d
Unified migration logging styles
no issue

- all migrations should contain a logging message
- info for successful things, or warn if we deviated from what was
  expected for some reason
- also added some spacing to make them easier to read
2021-02-18 17:35:32 +00:00
Daniel Lockyer
bf960bcecb
Fixed typo in migration message
no issue

- simple rewording to clean up the message
2021-02-18 17:05:00 +00:00
Kevin Ansfield
c7a29b4e7c
🐛 Fixed unexpected members-only content appearing in excerpt field (#12670)
closes https://github.com/TryGhost/Team/issues/468

- updated post-gating
  - clears excerpt if there's no access
  - rebuilds excerpt from free preview if paywall card is used and there's no custom excerpt
2021-02-18 14:12:06 +00:00
Rish
6d2b731379 Fixed member status events query
refs https://github.com/TryGhost/Ghost/issues/12602

- Adds aggregate option to permitted options
- Cleans up SQL query to fetch delta values
2021-02-18 18:11:50 +05:30
Fabien 'egg' O'Carroll
7abc6ac705
Added initial member events timeline API (#12661)
refs https://github.com/TryGhost/Ghost/issues/12633

Allows us to show a list of recent member events for the site dashboard
2021-02-18 11:52:18 +00:00
Kevin Ansfield
2c96df42ac
Added paywall card to mark end of free content preview (#12663)
closes https://github.com/TryGhost/Team/issues/466

- upgraded kg-default-cards to include paywall card
- extracted `htmlToPlaintext` from post model to shared util for re-use
- updated post-gating to set html+plaintext to the free preview if a paywall card has been used
  - re-generates plaintext from the truncated html using `htmlToPlaintext` util
- display free content in the `{{content}}` helper via the default CTA template
2021-02-17 23:00:26 +00:00
Naz
abb8c1df74 Updated unsplash setting migration to be idempotent
refs https://github.com/TryGhost/Ghost/issues/10318

- Previous version of migration was not following the idempotence rule of migrations
2021-02-18 10:55:59 +13:00
Naz
e0cc314f91 Removed duplicate migration
no issue

- The migration didn't want to go away after rebases combined with renames
2021-02-18 10:55:59 +13:00
Fabien O'Carroll
7785fb6dc5 Refactored usage of pre v4 "unsplash" setting format
refs https://github.com/TryGhost/Ghost/issues/10318

- API changes introduced:

canary/v4 Admin API
GET /settings/ (browse)

+ "unsplash" present in response as boolean value

GET /settings/:settingName (read)

+ "unsplash" present in response as boolean value

PUT /settings/ (edit)

+ "unsplash" updates setting, accepts ONLY  boolean format

v3  Admin API
GET /settings/ (browse)

+ "unsplash" present in response with object value

GET /settings/:settingName (read)

+ "unsplash" present in response with object value

PUT /settings/ (edit)

+ "unsplash" updates setting, accepts either boolean or object formats

v2 Admin API
GET /settings/ (browse)

+ "unsplash" present in response with object value

GET /settings/:settingName (read)

+ "unsplash" present in response with object value

PUT /settings/ (edit)

+ "unsplash" updates setting, accepts object format
2021-02-18 10:55:59 +13:00
Fabien O'Carroll
9cb069eb8e 🏗 Updated "unsplash" setting handling to match updated format
refs https://github.com/TryGhost/Ghost/issues/10318

- JSON object format used in previous "unsplash" setting was considered an
anti-pattern. Flat structure was extracted out of the "unsplash.isActive" JSON.
- The naming convention uses `amp` as  a precedent (https://github.com/TryGhost/Team/issues/331#issuecomment-658815017)
2021-02-18 10:55:59 +13:00
Naz
0097f2037c Fixed slack setting migration to follow idempotance rule
refs https://github.com/TryGhost/Ghost/pull/12631

- Initialy writen migration would break if executed twice
- Also accounted for situation if only part of the migration executed leaving only one of the values migrated
2021-02-18 10:55:32 +13:00
Hannah Wolfe
d3b50c8af3 Updated portal to include accent color in preview mode
- If we are in preview mode, we output an additional string to set the accent color for portal
- This is semi-temporary, as we will probably have a proper way to pass portal settings later
2021-02-17 13:49:41 +00:00
Hannah Wolfe
a0bdba2516 Added theme preview mode
- Allow the frontend to accept post messages to generate previews of the frontend
- Created a new endpoint in admin we can use to render these previews, which is possibly not necessary
- Supports a limited group of settings, which can easily be expanded, but care should be taken if expanding to use user-provided strings
2021-02-17 13:49:41 +00:00
Naz
d0e0760dae Hardcoded labs to always return members:true
refs https://github.com/TryGhost/Ghost/issues/10318

- Because members is effectively "enabled" by default starting Ghost 4.0 have hardcoded labs setting to be such. The alternative of removing this key from labs would be equivalent to `labs.members === false` which is undesireable and would mean additional work on theme developer's side.
2021-02-17 19:55:16 +13:00
Naz
37ef40b46e Dropped support for labs setting parameter
refs https://github.com/TryGhost/Ghost/issues/10318

- `labs` setting is dropped from setting values as the use of JSON objec
to sore settings has been deprecated
- `labs` setting is no longer accepted as a paramter in the Settings API nor the
impoprter. The value is ignored if present in the POST/PUT requests and
returns 404 in case it is requested by key at `GET /settings/:key`
2021-02-17 19:55:16 +13:00
Naz
ea6d656457 🏗 Removed labs from setting
refs https://github.com/TryGhost/Team/issues/332

- The last value that has been used in the code was "members"
- By default members will be always "on" starting Ghost 4.0, so there's no need for this flag anymore
- Therefore there's no real need to keep "labs" around
2021-02-17 19:55:16 +13:00
Thibaut Patel
58be6d23db Split the "Too many requests" error in ping service
issue https://github.com/TryGhost/Team/issues/362
2021-02-17 13:18:45 +13:00
Daniel Lockyer
1f6df7f293
Updated Ghost-Admin to v4.0.0-alpha.2 2021-02-16 19:53:52 +00:00
Kevin Ansfield
cabf718ae1
Added migration to transform all post content urls from absolute to relative (#12658)
refs https://github.com/TryGhost/Team/issues/467
refs https://github.com/TryGhost/Team/issues/221

- we added robust url transformation in Ghost 3.0 which ensures newly created or edited content is stored with relative URLs in the database
- this migration loops over all posts (and posts_meta) in the database performing the absolute->relative transformation to normalise data for old posts
2021-02-16 19:00:04 +00:00
Fabien O'Carroll
037b2ce761 Renamed migration to fill 01
no-issue
2021-02-16 18:19:09 +00:00
Fabien O'Carroll
87471fd5c6 Revert "Updated plan_nickname column to be nullable"
This reverts commit 5d122c3c62.
2021-02-16 18:19:09 +00:00
Fabien 'egg' O'Carroll
27b9e4ae04
Added member relation to events (#12656)
refs https://github.com/TryGhost/Ghost/issues/12602

When listing site-wide event, we want to include member information so
that we can contextualise the event when displaying it.
2021-02-16 16:14:20 +00:00
Kevin Ansfield
3dfbb6296b Fixed duplicate migration number 17 2021-02-16 12:33:24 +00:00
KiraLT
ba1ffb7b5f
🐛 Fixed EADDRINUSE error handling on NodeJS >=13 (#12591)
closes https://github.com/TryGhost/Ghost#12562

- From NodeJS v13 `error.errno` returns error code instead of a string. Because  of that use friendly "port is already in use" message did not work anymore. 
- Changed to use `error.code` which acts the same way as `error.errno` in older NodeJS versions.
2021-02-16 12:26:06 +00:00
Kevin Ansfield
66fe678cb7
Added 4.0 mobiledoc update migration (#12648)
refs https://github.com/TryGhost/Ghost/issues/12646

- adds `ghostVersion` property to mobiledoc of all posts so that we can pin the rendering output of existing content to protect against future breaking changes
- cleans any deprecated markdown card names from old Ghost 1.0 content
- cleans up HR cards that had an [accidental payload added](610d801bcf)
2021-02-16 12:13:30 +00:00
Kevin Ansfield
2d091fa8f9 Set mobiledoc.ghostVersion: '3.0' if missing on imported content
refs https://github.com/TryGhost/Ghost/issues/12646

- if the version is missing then the content was created before Ghost 4.0
- setting the version to `'3.0'` means it will continue to use the same rendering output so there are no unexpected breaking changes when migrating content
2021-02-16 12:08:40 +00:00
Fabien 'egg' O'Carroll
f4cb5c57c6
Updated members_status_events table (#12647)
refs https://github.com/TryGhost/Ghost/issues/12602

* Updated members_status_events table

By replacing the `status` column with a `from_status` and `to_status`
column, we are able to track the changes between multiple statuses
easier, and accumulate the data. e.g. the delta of paid members in a
given time range is the sum of the `to_status` columns set to 'paid'
minus the sum of the `from_status` columns set to 'paid' within that
time range

* Updated MEGA to handle addition of 'comped' status

With the addition of the 'comped' status, we need to ensure that MEGA
will still send emails to the correct recipients. I've opted to use an
"inverse" filter, as that is the intention of the free/paid split in
MEGA - as far as MEGA is concerned, "free" is the opposite of "paid"

* Updated customQuery for MemberStatusEvent

With the `status` column replaced with `from_status` and `to_status`
this allows us to fix and update the customQuery to correctly accumulate
the data into deltas over time, broken down by day.

* Populated members_status_events table

As the table will be used to generate deltas, we need to backfill the
data so that existing sites will be able to sum up the deltas and
calculate correct data.

The assumptions used in backfilling is that a Member's current status,
is their only status.
2021-02-16 10:38:36 +00:00
Fabien 'egg' O'Carroll
da9cd3b9d6
Updated status population to handle comped status (#12651)
refs https://github.com/TryGhost/Ghost/issues/12602

As part of collecting Member event data, we have added a third status
for members "comped" - this fixes the population of the column to handle
this
2021-02-16 10:37:06 +00:00
Rish
964fe222be Fixed incorrect volume delta selection in query
refs https://github.com/TryGhost/Ghost/issues/12602

The gross volume aggregate query was incorrectly naming the selected `SUM(amount)` value as `gross_volume` instead of expected `volume_delta`
2021-02-16 11:18:58 +05:30
Rish
0719b0618c Added new members stats endpoints
refs https://github.com/TryGhost/Ghost/issues/12633

Adds new stats endpoints for showing member dashboard, which fetches aggregate date over member events tables to form relevant datasets -

- Adds new endpoint for tracking MRR of site over time - `/members/stats/mrr`
- Adds new endpoint for tracking subscribers on site over time - `/members/stats/subscribers`
- Adds new endpoint for tracking gross volume of site over time - `/members/stats/gross_volume`
- Adds new endpoint for tracking member count on site over time - `/members/stats/count`
2021-02-16 11:18:58 +05:30
Hannah Wolfe
e0354ea5a0 Fixed attempts to stop server that isn't listening
- if our http server isn't listening yet, stop will throw an error
- also changeded the stopServer code to prefer a more readable syntax
2021-02-15 19:17:25 +00:00
Kevin Ansfield
ef76279084 Added mobiledoc.ghostVersion to blank mobiledoc document
refs https://github.com/TryGhost/Ghost/issues/12646

- when a blank mobiledoc document is created, ensure the correct Ghost version is added so content output is pinned to the version of Ghost that created it
2021-02-15 18:10:56 +00:00
Hannah Wolfe
21bea79848 Fixed inactive themes not loading [WIP]
- Currently the new boot process breaks the loading of inactive themes because the theme loader event isn't wired until after the server.start event is fired
- This is a WIP fix, there needs to be some bigger refactoring of the boot process once the old process is gone
2021-02-15 17:19:21 +00:00
Kevin Ansfield
70486ccb4e
Updated importer to transform 'card-markdown' to 'markdown' (#12650)
refs https://github.com/TryGhost/Ghost/issues/12646

- `card-markdown` support is being removed from the default mobiledoc cards so we need to make sure imported content doesn't fail
- removed long-unused `cardName` payload property from tests
2021-02-15 16:42:01 +00:00
Fabien O'Carroll
8880cc82c7 Refactored usage of the pre v4 "slack" setting
refs https://github.com/TryGhost/Ghost/issues/10318

- Change/compatibility table for this changeset:

Importer

+ "slack_username"  import
- "slack_url" ignore
+ "slack.username" import
- "slack.url" ignore

v3 & canary/v4 Admin APIs
GET /settings/ (browse)

+ "slack_username" present in response
+ "slack_url" present in response
+ "slack" present in response

GET /settings/:settingName (read)

+ "slack_username" present in response
+ "slack_url" present in response
+ "slack" present in response

PUT /settings/ (edit)

+ "slack_username" updates setting
+ "slack_url" updates setting
+ "slack" !NOTE! updates setting unless "slack_username" or "slack_ur"l are set (updated keys take priority)

v2 Admin API
GET /settings/ (browse)

- "slack_username" NOT present in response
- "slack_url" NOT present in response
+ "slack" present in response

GET /settings/:settingName (read)

- "slack_username" ignored (404)
- "slack_url" ignored (404)
- "slack" present in response

PUT /settings/ (edit)

- "slack_username" ignored (no error)
- "slack_url" ignored (no error)
+ "slack" updates setting
2021-02-15 15:25:54 +13:00
Fabien O'Carroll
12c28fe66e 🏗 Updated "slack" setting into "slack_{url,username}"
refs https://github.com/TryGhost/Ghost/issues/10318

- Object format used in previous "slack" setting was considered an
anti-pattern. Flag structure of separate slack_url and slack_username
values was extracted out of the "slack" JSON.
2021-02-15 15:25:54 +13:00
Rishabh Garg
aae2e68a1d
Added portal url to config (#12645)
no refs

The Portal URL that is shipped with every Ghost version is so far hardcoded in `ghost_head` and updated every-time we ship a new Portal minor/major change. This change brings the Portal URL inside the default Ghost config, which has few advantages -
- Allows easier access/managing of active Portal url/version
- Allows override for Portal URL for development/other purposes, where `config.*.json` allows using a Portal URL pointing to locally built copy for testing
2021-02-12 23:01:41 +05:30
Fabien O'Carroll
b5c1b9e4fa Added customQuery for MemberStatusEvent
refs https://github.com/TryGhost/Ghost/issues/12602

This allows us to build up a count of member statuses over time
2021-02-12 16:38:57 +00:00
Rishabh Garg
1d281581eb
Updated members_email_change_events to store previous email (#12641)
refs https://github.com/TryGhost/Ghost/issues/12602

As we only want to add events for when an email is changed, we need to store the previous email if we want a complete log of all emails for a member
2021-02-12 16:15:31 +00:00
Thibaut Patel
2f547b2974 Added the frontmatter column to meta
issue https://github.com/TryGhost/Team/issues/452
2021-02-12 15:31:04 +01:00
Fabien O'Carroll
905421b774 Populated members_paid_subscription_events table
refs https://github.com/TryGhost/Ghost/issues/12602

By backfilling the data, we allow exisiting sites to view their MRR over
time
2021-02-12 13:55:14 +00:00
Fabien O'Carroll
7cf0252392 Added MemberPaidSubscriptionEvent model
refs https://github.com/TryGhost/Ghost/issues/12602

- Adds the aggregateMRRDeltas option to the `findAll` method,
allowing us to calculate MRR over time
2021-02-12 13:55:14 +00:00
Fabien O'Carroll
a68d037cef Added MemberEmailChangeEvent model
refs https://github.com/TryGhost/Ghost/issues/12602
2021-02-12 13:55:14 +00:00