Commit Graph

162 Commits

Author SHA1 Message Date
Simon Backx
42ac8c41e4
🐛 Fixed uppercase file extensions ignored in content import (#14268)
refs https://github.com/TryGhost/Team/issues/1363

- When uploading a zip of images in Settings > Labs > [Import], it will skip images that have an uppercase extension, citing an 'unsupported file type' error.
- Cause: Glob ignored those files when matching extensions in ImportManager
- Fix: Added nocase option where needed
- Extended tests to also test the processZip method of ImportManager with getFilesFromZip
- Added isValidZip for zip with uppercase image
- Cleaned up JSDoc in ImportManager, and replaced some older JS syntax

Fixed zipContainsMultipleDataFormats error never thrown:

When a zip combines two data formats, no error was thrown.

- The promise error was only returned in an _.each loop, but never thrown
- Previously when combining multiple data types in a zip file, no error got thrown
- Added a test for this error
- Also added a test for noContentToImport error

Other errors and fixes:

- Added missing length in getBaseDirectory check
- getContentTypes fixed (returned duplicate values). Type error came up after adding all JSDocs
- updated tests to match real types from JSDoc and pass type validations
- Rewrote some methods in the async await syntax
- Added tests for ImportManager clean up
2022-03-11 09:17:58 +01:00
Naz
16e0736379 Removed Roon related importer code
no issue

- The support for the misformated roon imports was temporary anyway, based on the comments in code. It's also unecessary to keep around any code related to Roon as it's been _ages_ since anybody needed this kind of migration
2022-03-10 16:29:54 +08:00
Sam Lord
2887e416da
Switch to @tryghost/errors from ignition errors package (#13807)
refs: TryGhost/Toolbox#147

* Replaces all references to isIgnitionError with isGhostError
* Switches use of GhostError to InternalServerError - as GhostError is no longer public
There are places where InternalServerError is not the valid error, and new errors should be added to the @tryghost/errors package to ensure that we can use semantically correct errors in those cases.
2021-12-01 10:22:01 +00:00
Sam Lord
a5484a0fe1 Fixed transaction logic in data importer
https://github.com/TryGhost/Toolbox/issues/130

The transaction no longer commits in the promise chain, which wasn't
valid logic for a transaction, since it is commited automatically when
the promise chain resolves, and rollsback automatically when the
promise chain rejects.

This makes code which fails during the transaction error in the right
place, instead of getting stuck here. (Especially good for writing
tests).

The tests for this code can now live in the integration folder.
2021-11-16 17:09:55 +00:00
Naz
07afb08875 Made storage calls related to images use "images" feature
refs https://linear.app/tryghost/issue/CORE-1/multiple-adapters-per-type

- Having this preemptive change allows to separate implementation of "image" storage from future usecases like "videos", "audios" etc. Even if the "image" adapter is not configured the default behavior will fallback to use the "active" storage adapter. If there's a need to handle "images" differently through a custom apapter that'll work out of the box ;)
2021-10-21 20:22:45 +13:00
Hannah Wolfe
8b7d0fc6a1
Fixed linting error from bad merge
refs: #13380

- one of the prs I merged was a duplicate and ended up with the same messages object declared twice
2021-10-08 15:49:27 +01:00
Aleksander Chromik
cadd63bc9c
Replaced i18n.t w/ tpl in core/server/data (#13491)
refs: #13380

- The i18n package is deprecated. It is being replaced with the tpl package.

Co-authored-by: Aleksander Chromik <aleksander.chromik@footballco.com>
2021-10-08 15:31:17 +01:00
Alex Ward
189b7bb97b Replaced i18n.t w/ tpl helper in import-manager.js
refs: TryGhost#13380

The i18n package is deprecated. It is being replaced with the tpl package.
2021-10-07 11:31:44 +01:00
Naz
7589218abc Converted import-manager module to a class
refs https://github.com/TryGhost/Team/issues/694
refs https://linear.app/tryghost/issue/CORE-16/tackle-importersdataindexjs

- The codebase uses class syntax instead of extending/instantiating a native function (this is a very old of doing pseudo OOP in JS). Updated the old syntax in a very one-to-one brainless way with intention to improve the file again when touched again
2021-09-22 20:10:23 +12:00
Naz
c39d1996a4 Brought back importer index.js file exposing a single thing
refs https://github.com/TryGhost/Team/issues/694
refs https://linear.app/tryghost/issue/CORE-16/tackle-importersdataindexjs

- This extra step was done to keep the git history cleaner (previous commit was a clean rename). This new index file exposes just one thing.
- The next step here should be refactoring of the "importer-manager" module into a class following a DI pattern.
2021-09-22 20:10:23 +12:00
Naz
0d32552d7a Renamed imports indexjs to import-manager
refs https://github.com/TryGhost/Team/issues/694
refs https://linear.app/tryghost/issue/CORE-16/tackle-importersdataindexjs

- The index file should contain an API index that the module/folder exposes. In this instance it is a full blown class-like set of data and functions that don't belong to index.js
2021-09-22 20:10:23 +12:00
Naz
90e9e473ad Brought back index.js file exposing a single thing
refs https://github.com/TryGhost/Team/issues/694
refs https://linear.app/tryghost/issue/CORE-16/tackle-importersdataindexjs

- This extra step was done to keep the git history cleaner (previous commit was a clean rename). This new index file exposes just one thing.
- The next step here should probably be refactoring of the data-importer module into a class following a DI pattern.
2021-09-22 03:23:12 +12:00
Naz
208dd49971 Renamed imports/data indexjs to data-importer
refs https://github.com/TryGhost/Team/issues/694
refs https://linear.app/tryghost/issue/CORE-16/tackle-importersdataindexjs

- The index file should contain an API index that the module/folder exposes. In this instance it is a full blown class-like set of data and functions that don't belong to index.js
2021-09-22 03:23:12 +12:00
Fabien 'egg' O'Carroll
90a4d369db
Fixed imports for files missing the email_only key (#13284)
closes https://github.com/TryGhost/Team/issues/1024

Our importer would set the default value of all posts_meta keys to
`null`. This is an invalid value for the `email_only` key which only
accepts booleans.

Since we are already looping over the schema to create the default
values, we can use the `defaultTo` property in the schema to use the
intended default, and fall back to `null` if it doesn't exist.

We've used the `Reflect.has` function to determine if the `defaultTo`
key exists, as opposed to a truthy check, because it's possible that a
falsy value (e.g. false, in the case of email_only) can be used as the
default.
2021-09-06 11:51:42 +01:00
Hannah Wolfe
c29c118fcf
Moved labs utlity to shared
- This isn't really a "service" - it's a set of utilities for working with labs flags
- It's also required all over the place, and doesn't require anything that isn't shared
- Therefore, it should live in shared
2021-07-08 09:05:41 +01:00
Hannah Wolfe
8d38957bd7
Revert "Moved labs utlity to shared"
This reverts commit 782de52678.
2021-07-08 07:09:13 +01:00
Hannah Wolfe
782de52678
Moved labs utlity to shared
- This isn't really a "service" - it's a set of utilities for working with labs flags
- It's also required all over the place, and doesn't require anything that isn't shared
- Therefore, it should live in shared
2021-07-07 21:41:34 +01:00
Hannah Wolfe
526993965a
Switch to @trghost/validator, remove validator
- Part of the effort to split Ghost down into smaller, decoupled pieces
- Moved out our internal validator tooling to a separate library
- Replaced all usage of our own tooling and validatorjs directly with @tryghost/validator
- Removed the validatorjs dependency and removed the renovate pin
- This gives us a consistant, smaller, clearer public API for validations
- It will eventually be used on Ghost Admin too
- This way we can start getting up to date with validator whilst not increasing build size
2021-06-16 08:11:22 +01:00
Sam Lord
35e51e364b Switch to @tryghost/debug, remove ghost-ignition
no issue
The only pieces of Ghost-Ignition used in Ghost were debug and
logging. Both of these modules have been superceded by the Framework
monorepo, and all usages of Ignition have now been removed, replaced
with @tryghost/debug and @tryghost/logging.
2021-06-15 17:24:22 +01:00
Sam Lord
caea330647 Change to use @tryghost/logging
no issue

Logging is now controlled by a logginrc.js file in the root of the project - and now we can just import @tryghost/logging everywhere
2021-06-15 15:59:11 +01:00
Hannah Wolfe
0fe8426f97
Renamed validation to validator + better public API
- renamed our internal validation library to "validator" - which is the same as the tool it wraps
- updated the public api so that validator methods are directly exposed
- this will make it a drop-in replacement for validator-js
- in turn, this allows us to pull this out into @tryghost/validator, and use our own wrapper instead of the 3rd party library
2021-06-15 15:32:36 +01:00
Naz
d8230f3343 Added labs setting import validation
refs https://github.com/TryGhost/Team/issues/757

- Similarly to the API input serrialization importer should not pass through unknown labs keys to avoid soiling data
2021-06-08 02:18:17 +12:00
Naz
49ba26373d 🏗 Added "labs" setting enabling feature flags
refs https://github.com/TryGhost/Team/issues/757
refs https://github.com/TryGhost/Team/issues/332
refs ea6d656457

- We have a need a quick way to add features behind flags. The old way of "labs" is the quickest way to achieve this. It has ready tooling around it and well understood pitfalls. This change reintroduces "labs" group & key in settings table in the same shape it used to be (see reffed commit)
- Next step will be introducing very basic guard rails to protect from pitfalls previous implementation of "labs" had. This will include an allowlist based input validation for lab's object's data
-  The labs being an "object" type is an EXCEPTION. Even though it's an antipattern we aim to move away from, for now it's the lowest impact solution that will unblock the use of flags in the system. A proper solution will come at some point.
2021-06-08 02:18:17 +12:00
Hannah Wolfe
273e220327 Moved i18n to shared
refs 829e8ed010

- i18n is used everywhere but only requires shared or external packages, therefore it's a good candidate for living in shared
- this reduces invalid requires across frontend and server, and lets us use it everywhere until we come up with a better option
2021-05-04 13:03:38 +01:00
Hannah Wolfe
829e8ed010 Expanded requires of lib/common i18n and events
- Having these as destructured from the same package is hindering refactoring now
- Events should really only ever be used server-side
- i18n should be a shared module for now so it can be used everywhere until we figure out something better
- Having them seperate also allows us to lint them properly
2021-05-03 17:14:52 +01:00
Daniel Lockyer
e9b21fdbd1 Updated bson-objectid calls to match API change
refs c873899e49

- as of `bson-objectid` v2.0.0, this library exports the function
  to generate an ObjectID directly, and then you need to use `.toHexString()`
  to get the 24 character hex string - 6696f27d82
- this commit removes all uses of `.generate()` and replaces with this
  change
2021-04-21 16:23:52 +01:00
Kevin Ansfield
fff6a04c54
Migrated members_allow_free_signup setting to members_signup_access (#12886)
refs https://github.com/TryGhost/Team/issues/579

Currently the members signup setting is explicitly yes/no to allowing free members signup, with the implication that when set to "no" members is still active but members have to be created via Stripe or the admin API.

This change renames the setting and changes its type to allow more than a binary option.

- migration to create/update the new setting based on the old value
  - free signup = "all", no free signup = "invite"; matches the current UI for this setting
- rename setting everywhere it's used/tested against
- modify `getAllowSelfSignup()` used to configure members packages to only return `true` when the new setting is set to `'all'` to match behaviour to the older setting
- update importer to rename the setting when importing from an older Ghost version
2021-04-19 16:36:30 +01:00
Fabien 'egg' O'Carroll
15978689c0
Added default value to accent_color setting (#12743)
refs TryGhost/Team#535

We want to ensure that a site will always have a default value of
`'#15171A'` for the accent_color setting.

Since the boot process changed we have three cases to account for:

1. Setting does not exist
2. Setting exists with no value
3. Setting exists with a value

It is only in the case of 2. that we want the migration to update the
database with a default value.

In the case of 3. the site owner has already set a value, which we do
not want to override.

In the case of 1. the setting will be created (and populated with
default value) from the default-settings.json file, by the
populateDefaults method called from the settings service

We also update the accent_color setting to include a non-empty
validation, to ensure that the setting will always have a value, as
sites before 4.x may have an empty accent_color, we must update the
importer to set the default value if one is not present. Otherwise we
would run into validation errors and even if we didn't would have an
invalid database state.
2021-03-09 16:15:10 +00:00
Hannah Wolfe
e30b9735fa Added staff user limit
refs: https://github.com/TryGhost/Team/issues/510

- In the case that host config is provided, keep staff users within the limiti
- The definition of a staff user is a user with a role other than Contributor, and whose status is not inactive
   - Contributors don't count
   - Suspended (status inactive) users don't count
   - Locked users DO count
   - Invited users DO count
- You can't invite more staff users whilst there are pending invites
- You can't unsuspend a user, or change the role on a user in such a way as will take you over your limit
- You can't import staff users - all imported users are automatically set to Contributors
- As part of this work, we are changing the default Ghost user to a Contributor otherwise it uses up a staff user

Note: there is one known active bug with this commit.
- Assume you have one remaining user within your limit. You send an invite, this works.
- You cannot "resend" that invite, it will think you're sending a new invite and hit the limit
- You must "revoke" that invite first, and create a new one
- This bug exists because the resend function uses the add endpoint & does a delete+add, but this hits the permission check before the delete
2021-03-04 16:13:04 +00:00
Fabien O'Carroll
7785fb6dc5 Refactored usage of pre v4 "unsplash" setting format
refs https://github.com/TryGhost/Ghost/issues/10318

- API changes introduced:

canary/v4 Admin API
GET /settings/ (browse)

+ "unsplash" present in response as boolean value

GET /settings/:settingName (read)

+ "unsplash" present in response as boolean value

PUT /settings/ (edit)

+ "unsplash" updates setting, accepts ONLY  boolean format

v3  Admin API
GET /settings/ (browse)

+ "unsplash" present in response with object value

GET /settings/:settingName (read)

+ "unsplash" present in response with object value

PUT /settings/ (edit)

+ "unsplash" updates setting, accepts either boolean or object formats

v2 Admin API
GET /settings/ (browse)

+ "unsplash" present in response with object value

GET /settings/:settingName (read)

+ "unsplash" present in response with object value

PUT /settings/ (edit)

+ "unsplash" updates setting, accepts object format
2021-02-18 10:55:59 +13:00
Naz
ea6d656457 🏗 Removed labs from setting
refs https://github.com/TryGhost/Team/issues/332

- The last value that has been used in the code was "members"
- By default members will be always "on" starting Ghost 4.0, so there's no need for this flag anymore
- Therefore there's no real need to keep "labs" around
2021-02-17 19:55:16 +13:00
Kevin Ansfield
2d091fa8f9 Set mobiledoc.ghostVersion: '3.0' if missing on imported content
refs https://github.com/TryGhost/Ghost/issues/12646

- if the version is missing then the content was created before Ghost 4.0
- setting the version to `'3.0'` means it will continue to use the same rendering output so there are no unexpected breaking changes when migrating content
2021-02-16 12:08:40 +00:00
Kevin Ansfield
70486ccb4e
Updated importer to transform 'card-markdown' to 'markdown' (#12650)
refs https://github.com/TryGhost/Ghost/issues/12646

- `card-markdown` support is being removed from the default mobiledoc cards so we need to make sure imported content doesn't fail
- removed long-unused `cardName` payload property from tests
2021-02-15 16:42:01 +00:00
Fabien O'Carroll
8880cc82c7 Refactored usage of the pre v4 "slack" setting
refs https://github.com/TryGhost/Ghost/issues/10318

- Change/compatibility table for this changeset:

Importer

+ "slack_username"  import
- "slack_url" ignore
+ "slack.username" import
- "slack.url" ignore

v3 & canary/v4 Admin APIs
GET /settings/ (browse)

+ "slack_username" present in response
+ "slack_url" present in response
+ "slack" present in response

GET /settings/:settingName (read)

+ "slack_username" present in response
+ "slack_url" present in response
+ "slack" present in response

PUT /settings/ (edit)

+ "slack_username" updates setting
+ "slack_url" updates setting
+ "slack" !NOTE! updates setting unless "slack_username" or "slack_ur"l are set (updated keys take priority)

v2 Admin API
GET /settings/ (browse)

- "slack_username" NOT present in response
- "slack_url" NOT present in response
+ "slack" present in response

GET /settings/:settingName (read)

- "slack_username" ignored (404)
- "slack_url" ignored (404)
- "slack" present in response

PUT /settings/ (edit)

- "slack_username" ignored (no error)
- "slack_url" ignored (no error)
+ "slack" updates setting
2021-02-15 15:25:54 +13:00
Naz
d254a697b9 🏗 Cleanedup app-related settings
refs https://github.com/TryGhost/Ghost/issues/12565
refs f74d459dad

- These fields should have been cleaned up as a result of changed done in referenced commit
2021-02-08 12:25:52 +13:00
Aileen Nowak
c9d1fa0b96 Updated links to ghost.org sites
no issue

Follow-up task of the updated Ghost Docs structure. Updated links reflecting the new structure to prevent unnecessary 404s and redirects.
2021-01-20 09:59:45 +13:00
Fabien 'egg' O'Carroll
4604ba1587
Fixed backward compatibility for send_email_when_published (#12357)
no-issue

* Handled send_email_when_published in Posts API

This restores backwards compatibility of the Posts API allowing existing
clients to continue to use the `send_email_when_published` flag. This
change uses two edits, which is unfortunate. The reason being is that
this is an API compatibility issue, not a model issue, so we shouldn't
introduce code to the model layer to handle it. The visibility property
of the model is used to determine how to fall back, and because it can
be left out of the API request, and relies on a default in the settings,
we require that the model decide on the `visibility` before we run our
fallback logic (or we duplicate the `visibility` default at the cost of
maintenance in the future)

* Dropped send_email_when_published column from posts

Since this column is not used any more, we can drop it from the table.
We include an extra migration to repopulate the column in the event of
a rollback

* Updated importer to handle send_email_when_published

Because we currently export this value from Ghost, we should correctly
import it. This follows the same logic as the migrations for this value.

* Included send_email_when_published in API response

As our v3 API documentation includes `send_email_when_published` we must
retain backward compatibility by calculating the property.

* Fixed fields filter with send_email_when_published

* Added safety checks to frame properties

Some parts of the code pass a manually created "frame" which is missing
lots of properties, so we check for the existence of all of them before
using them.

* Fixed 3.1 migration to include columnDefinition

We require that migrations have all the information they need contained
within them as they run in an unknown state of the codebase, which could
be from the commit they are introduced, to any future commit. In this
case the column definition is removed from the schema in 3.38 and the
migration would fail when run in this version or later.
2020-11-11 13:03:41 +00:00
naz
38b7f8e311
Fixed "no-shadow" linting error in server/data modules (#12288)
refs 143921948d

- Continuation of changes started in referenced commit
2020-10-20 11:56:46 +13:00
Rish
a5a7515fe8 Fixed typo in private setting import
no refs

Last commit introduced a small typo in how private setting is ignored during import
2020-10-15 15:55:24 +05:30
Rish
3e0948d04e Ignored member email settings in import
no issue

- Member email settings - newsletter and support - was previously being set to null for all cases, which can lead to `null` values being set for those addresses on a successful import whereas the expected behavior is to retain the old values.
- Adds the 2 email settings to ignored settings list so they are not changed during import process
2020-10-15 15:32:05 +05:30
Rish
cbb3664bdd Added handling for new members support/reply email addresses
no issue

- Updated magic link generation and validation methods for email update API to handle new support address
- Updated importer to ignore the new support address as it can only be updated via verification
- Updated members service to listen on settings edit for new support/reply address fields as well
- Updated tests to include the new settings
2020-08-31 18:09:38 +05:30
Daniel Lockyer
5b471e1bbe Extracted promise libs and history into @tryghost/promise
- deleted files under `core/server/lib/promise` and related test files
- added `@tryghost/promise` as a dependency
- fixed all local requires to point to the new package
2020-08-11 18:44:21 +01:00
Fabien 'egg' O'Carroll
37d577fa90
Remove references to bulk email settings (#12000)
refs #10318

* Ignored bulk_email_settings when importing
* Removed custom handling for bulk_email_settings
* Removed bulk_email_settings from default-settings
* Ignored bulk_email_settings when editing settings
2020-07-03 16:30:54 +02:00
Rish
212e39f772 Updated settings importer to handle new mapping
refs https://github.com/TryGhost/Ghost/issues/10318

- Skipped `members_subscription_settings` and `stripe_connect_integration` from settings import
- Fixed import data field filtering for core and theme groups
2020-06-30 17:38:36 +05:30
Fabien 'egg' O'Carroll
ee786aaa5d
Cleaned up members & stripe settings (#11957)
* Updated members default settings

ref #10318

This pulls out the members_subscription_settings & stripe_connect_intgration settings into separate keys

* Updated usage of members_from_address

* Updated stripe_connect usage

* Updated members config to use new settings

* Updated members middleware to use isStripeConnected

* Updated members service to reload correctly

We reload the members-api instance when the related settings change, so
this makes sure we're listening to the correct settings changes

* Updated ghost_head helper to use new settings

* Updated theme middleware to use new settings

* Renamed members_allow_signup -> members_allow_free_signup

* Fixed tests after settings refactor

* Removed  from direct key settings key

* Fixed regression tests for settings api
2020-06-29 16:22:42 +02:00
Nazar Gargol
8ea245fc7c Fixed importer mapping for renamed default_locale, active_timezone, ghost_* settings
refs #10318
refs 2614565d5a

- Adds importer mapping for fields as in referenced migration
- The intention is to allow exports from v2/v3 to still be compatilbe with current version
- Adds a mapper for deprecated fields and imports them with new settings "keys"
2020-06-24 14:19:52 +12:00
Kevin Ansfield
8a817050c5
Organised default settings into new groups (#11952)
refs https://github.com/TryGhost/Ghost/issues/10318

- precursor to migrating from `settings.type` to `settings.group`
- renames `blog` type to `site`
- renames `bulk_email` type to `email`
- moves settings out of `site` (previously `blog`) into more appropriate groups such as `core` or individual feature groups
2020-06-23 20:49:08 +01:00
Nazar Gargol
b78dbfbbc2 Removed use of deprecated setting flags from the codebase
refs #10318
refs https://github.com/TryGhost/Ghost/pull/11942

- Removes force_i18n, permalinks, and members_session_secret usage from the codebase
-  We deprecated these flasgs and have not used since Ghost v2. It's good time to remove them before we introduce bigger changes to how `settings` table opeartes.
- Fixed importer test. The test was meant to check if string values were converted properly, the check agains boolean didn't make much sense in this context, so removed it.
- Following this change are going to come ralated migrations to fix existing data (see ref)
2020-06-23 11:58:19 +12:00
Rishabh Garg
84d0a46c6e
Removed "from address" overwrite in importer (#11914)
refs https://github.com/TryGhost/Ghost/issues/11414

- Importing data currently overwrites the existing "from address" with new value
- "from address" needs to go through email validation flow before update which was bypassed
- Updates importer to not allow overwrite for "from address" and use existing
- Adds test for "from address" overwrite
2020-06-15 12:22:55 +05:30
Vikas Potluri
00c324fa4e
Moved core/server/lib/common/logging to core/shared/logging (#11857)
- Represents that logging is shared across all parts of Ghost at present
  * moved core/server/lib/common/logging to core/shared/logging
  * updated logging path for generic imports
  * updated migration and schema imports of logging
  * updated tests and index logging import
  * 🔥 removed logging from common module
  * fixed tests
2020-05-28 19:30:23 +01:00