Commit Graph

33086 Commits

Author SHA1 Message Date
Naz
116e69afcd
Refactored importer's content file handler
refs https://github.com/TryGhost/Toolbox/issues/523

- Renamed variables that were too media-specific
2023-03-02 17:54:19 +08:00
Naz
cd49992cd9
Added generic file import handler
refs https://github.com/TryGhost/Toolbox/issues/523
refs c2534e3c86/packages/mg-assetscraper/lib/AssetScraper.js (L14-L16)
refs https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types

- Importer needs to process and recognize document files like pdfs, presentations etc to be able to import them into sites file storage.
- The handler allows a new root directory "files" to place imported documents
- The handler adds validation and processing for following file extensions:
".pdf",
".json",
".jsonld",
".odp",
".ods",
".odt",
".ppt",
".pptx",
".rtf",
".txt",
".xls",
".xlsx",
".xml"
- With following content types:
"application/pdf",
"application/json",
"application/ld+json",
"application/vnd.oasis.opendocument.presentation",
"application/vnd.oasis.opendocument.spreadsheet",
"application/vnd.oasis.opendocument.text",
"application/vnd.ms-powerpoint",
"application/vnd.openxmlformats-officedocument.presentationml.presentation",
"application/rtf",
"text/plain",
"application/vnd.ms-excel",
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
"application/xml",
"application/atom+xml"
2023-03-02 17:54:18 +08:00
Naz
b0b80d672d
Refactored content file handler constructor
refs https://github.com/TryGhost/Toolbox/issues/523

- To support generic "file" import handling the content file handler needs more configuration options for properties like type, extensions, content path etc. This refactor makes the handler configurable and reusable for any type of file import
2023-03-02 17:54:18 +08:00
Naz
bf215be0f8
Renamed media import handler to file handler
refs https://github.com/TryGhost/Toolbox/issues/523

- We need to process generic files like .pdf, .md, etc. on the importer "handler" stage.
- The media handler can process more than just media files after few refactorings. Renaming it to a generic content file handler indicates it can process any type of content file.
- In future we can substitute the built-in "images" import handler with this generic content file handler.
2023-03-02 17:54:18 +08:00
renovate[bot]
caac055bdc Update mock-knex digest to d8b93b1 2023-03-02 10:17:13 +01:00
Daniel Lockyer
6460522352
Pinned dependencies
- these should be pinned because Ghost is an application and we like to
  know what versions of dependencies we're running
2023-03-02 09:47:16 +01:00
renovate[bot]
9f08732039 Update @tryghost 2023-03-02 09:47:02 +01:00
Ronald Langeveld
7c805b7176 Revert "Added new created tier e2e test"
This reverts commit dc98868961.
2023-03-02 16:39:17 +08:00
Barno
389362f463
Replaced Promise.props() function with .all() in Members stats
refs https://github.com/TryGhost/Ghost/issues/14882

- this helps remove uses of Bluebird within Ghost

Co-authored-by: Carol-Barno <cbarno@innovexsolutions.co.ke>
2023-03-02 09:28:36 +01:00
Ronald Langeveld
dc98868961 Added new created tier e2e test
ref https://github.com/TryGhost/Team/issues/1446

- Added e2e test to check if a new tier can be added successfully.
2023-03-02 16:11:12 +08:00
renovate[bot]
e01d8a6f1d Update sentry-javascript monorepo to v7.40.0 2023-03-02 09:07:07 +01:00
Naz
59d0a1dd71
Added a comment about strict content/media imports
refs https://github.com/TryGhost/Toolbox/issues/523

- Making "content/media" a strict requirement for the import folder structure breaks the importer (glob library does not expand a subdirectory pattern). Unless it become a strict requirement we can use general content directory matching in combination with file extensions.
2023-03-02 14:35:37 +08:00
Naz
0c111cfe6b
Refactored media file import path calculation
refs https://github.com/TryGhost/Toolbox/issues/523

- The naming was referring to "image", which is a leftover from a copy-paste.
- It's much nicer to read a skimmable, columnar urlJoin method parameters instead of one long line
2023-03-02 14:35:37 +08:00
Naz
2361669bb6
Added media file support to the zip importer
refs https://github.com/TryGhost/Toolbox/issues/523

- When migrating or importing ZIP files into Ghost there's often a need to include more than just post content and images.
- When media files are present in the imported zip file they are now copied across and processed along with the rest of import files: json, images, csvs, etc.
- The importer also searches for use of the media files in the imported "posts" substituting the links with local ones
- The media importer recognizes media files inside of "media" or "content/media" folders present in the zip. The supported media file extensions are same as for media upload widget:
".mp4", ".webm", ".ogv", ".mp3", ".wav", ".ogg", ".m4a"

with following content-types:
 "video/mp4", "video/webm", "video/ogg", "audio/mpeg", "audio/vnd.wav", "audio/wave", "audio/wav", "audio/x-wav", "audio/ogg", "audio/mp4", "audio/x-m4a"
2023-03-02 14:35:37 +08:00
Naz
49553e5942
Renamed "image" importer to generic ContentFileImporter
refs https://github.com/TryGhost/Toolbox/issues/523

- This is ground work before introducing a "media" content type importer
- Previous "image" file name was not describing well what the importer was capable of doing
2023-03-02 14:35:37 +08:00
Ronald Langeveld
6361f27ec1
🐛 Fixed broken link filter params (#16351)
closes https://github.com/TryGhost/Team/issues/2403

- Fixes the links in `list-item.hbs` rendereing the url filter params with brackets
eg `['postid']` where the filter UI requires the filter params not to
have brackets to allow it to render with the filter pre-selected.
- The API handles both with and without brackets.
2023-03-02 13:45:44 +08:00
Steve Larson
3a9adcbc01 updated external request format
refs TryGhost/Team#2663
-got version upgrade broke this request
-moved to new format
2023-03-01 14:51:42 -06:00
Daniel Lockyer
a9c0a64fd2
Merged package .gitignore files into top-level file
fixes https://github.com/TryGhost/Toolbox/issues/480

- this is a bit tricky because the files are a mess but I think this
  covers everything and deduplicates having multiple .gitignore files
  across the repo so we can focus on keeping the definitions in one
2023-03-01 18:31:05 +01:00
Sam Lord
90ef8f008e Prevent error logs from occuring in Ghost unit test suite
refs: https://github.com/TryGhost/Toolbox/issues/389
2023-03-01 16:37:46 +00:00
renovate[bot]
dd0a3be7e9
Update dependency @babel/plugin-proposal-decorators to v7.21.0 (#16161)
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
|
[@babel/plugin-proposal-decorators](https://babel.dev/docs/en/next/babel-plugin-proposal-decorators)
([source](https://togithub.com/babel/babel)) | [`7.20.7` ->
`7.21.0`](https://renovatebot.com/diffs/npm/@babel%2fplugin-proposal-decorators/7.20.7/7.21.0)
|
[![age](https://badges.renovateapi.com/packages/npm/@babel%2fplugin-proposal-decorators/7.21.0/age-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://badges.renovateapi.com/packages/npm/@babel%2fplugin-proposal-decorators/7.21.0/adoption-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://badges.renovateapi.com/packages/npm/@babel%2fplugin-proposal-decorators/7.21.0/compatibility-slim/7.20.7)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://badges.renovateapi.com/packages/npm/@babel%2fplugin-proposal-decorators/7.21.0/confidence-slim/7.20.7)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>babel/babel</summary>

###
[`v7.21.0`](https://togithub.com/babel/babel/blob/HEAD/CHANGELOG.md#v7210-2023-02-20)

[Compare
Source](https://togithub.com/babel/babel/compare/v7.20.13...v7.21.0)

##### 🚀 New Feature

- `babel-core`, `babel-helper-create-class-features-plugin`,
`babel-plugin-proposal-class-properties`,
`babel-plugin-proposal-private-methods`,
`babel-plugin-proposal-private-property-in-object`
- [#&#8203;15435](https://togithub.com/babel/babel/pull/15435) feat:
Implement `privateFieldsAsSymbols` assumption for classes
([@&#8203;fwienber](https://togithub.com/fwienber))
- `babel-helper-create-regexp-features-plugin`,
`babel-plugin-proposal-regexp-modifiers`, `babel-standalone`
- [#&#8203;15226](https://togithub.com/babel/babel/pull/15226) feat:
Support regexp modifiers proposal
([@&#8203;liuxingbaoyu](https://togithub.com/liuxingbaoyu))
- `babel-cli`, `babel-core`, `babel-generator`,
`babel-plugin-transform-destructuring`,
`babel-plugin-transform-modules-commonjs`,
`babel-plugin-transform-react-jsx`, `babel-traverse`
- [#&#8203;15022](https://togithub.com/babel/babel/pull/15022) feat:
Generate sourcemaps of friendly call frames
([@&#8203;liuxingbaoyu](https://togithub.com/liuxingbaoyu))
-   `babel-parser`, `babel-types`
- [#&#8203;15384](https://togithub.com/babel/babel/pull/15384) \[ts]
Support `const` modifier in type parameters
([@&#8203;nicolo-ribaudo](https://togithub.com/nicolo-ribaudo))
- `babel-generator`, `babel-helpers`, `babel-parser`,
`babel-plugin-proposal-decorators`, `babel-plugin-syntax-decorators`,
`babel-runtime-corejs2`, `babel-runtime-corejs3`, `babel-runtime`
- [#&#8203;15405](https://togithub.com/babel/babel/pull/15405) Implement
decorators as presented at `2023-01` TC39 meeting
([@&#8203;nicolo-ribaudo](https://togithub.com/nicolo-ribaudo))
-   `babel-parser`
- [#&#8203;15114](https://togithub.com/babel/babel/pull/15114) Parser
option to allow `new.target` outside functions
([@&#8203;overlookmotel](https://togithub.com/overlookmotel))
- [#&#8203;15320](https://togithub.com/babel/babel/pull/15320) Add
`annexb: false` parser option to disable Annex B
([@&#8203;nicolo-ribaudo](https://togithub.com/nicolo-ribaudo))
-   `babel-core`
- [#&#8203;15283](https://togithub.com/babel/babel/pull/15283) feat:
Support `.cts` as configuration file
([@&#8203;liuxingbaoyu](https://togithub.com/liuxingbaoyu))
- `babel-generator`, `babel-parser`, `babel-plugin-transform-typescript`
- [#&#8203;15381](https://togithub.com/babel/babel/pull/15381) \[ts]
Support `export type * from`
([@&#8203;nicolo-ribaudo](https://togithub.com/nicolo-ribaudo))

##### 🐛 Bug Fix

-   `babel-plugin-transform-typescript`
- [#&#8203;15379](https://togithub.com/babel/babel/pull/15379) \[ts5.0]
Better inlining of constants in enums
([@&#8203;liuxingbaoyu](https://togithub.com/liuxingbaoyu))
-   `babel-core`
- [#&#8203;15366](https://togithub.com/babel/babel/pull/15366) handling
circular/shared structures in deep-clone
([@&#8203;azizghuloum](https://togithub.com/azizghuloum))
- `babel-helper-create-class-features-plugin`,
`babel-plugin-proposal-class-properties`,
`babel-plugin-proposal-class-static-block`,
`babel-plugin-proposal-private-methods`,
`babel-plugin-transform-classes`, `babel-plugin-transform-new-target`
- [#&#8203;15406](https://togithub.com/babel/babel/pull/15406) Preserve
class elements comments in class transform
([@&#8203;JLHwung](https://togithub.com/JLHwung))
- `babel-parser`, `babel-plugin-transform-flow-comments`,
`babel-plugin-transform-flow-strip-types`, `babel-types`
- [#&#8203;15414](https://togithub.com/babel/babel/pull/15414) \[ts] Fix
restrictions for optional parameters
([@&#8203;nicolo-ribaudo](https://togithub.com/nicolo-ribaudo))

##### 💅 Polish

-   `babel-parser`
- [#&#8203;15400](https://togithub.com/babel/babel/pull/15400) polish:
improve "`await` as identifier" error in modules
([@&#8203;JLHwung](https://togithub.com/JLHwung))

##### 🏠 Internal

-   `babel-core`
- [#&#8203;15137](https://togithub.com/babel/babel/pull/15137) Improve
CJS compat with ESM-based `@babel/core`
([@&#8203;nicolo-ribaudo](https://togithub.com/nicolo-ribaudo))

##### 🔬 Output optimization

-   `babel-plugin-transform-typescript`
- [#&#8203;15418](https://togithub.com/babel/babel/pull/15418) \[ts]
Handle exponentiation operator in constant folding
([@&#8203;ehoogeveen-medweb](https://togithub.com/ehoogeveen-medweb))

###
[`v7.20.13`](https://togithub.com/babel/babel/blob/HEAD/CHANGELOG.md#v72013-2023-01-21)

[Compare
Source](https://togithub.com/babel/babel/compare/v7.20.7...v7.20.13)

##### 🐛 Bug Fix

- `babel-helpers`, `babel-plugin-proposal-decorators`,
`babel-runtime-corejs2`, `babel-runtime-corejs3`, `babel-runtime`
- [#&#8203;15332](https://togithub.com/babel/babel/pull/15332) Ensure
class decorators can access decorated non-static members
([@&#8203;JLHwung](https://togithub.com/JLHwung))
-   `babel-plugin-transform-typescript`
- [#&#8203;15349](https://togithub.com/babel/babel/pull/15349) fix:
Preserve `import {type T} from 'x'` when `onlyRemoveTypeImports:true`
([@&#8203;liuxingbaoyu](https://togithub.com/liuxingbaoyu))
- [#&#8203;15344](https://togithub.com/babel/babel/pull/15344) fix:
Properties that are `abstract` should not be initialized.
([@&#8203;liuxingbaoyu](https://togithub.com/liuxingbaoyu))
-   `babel-parser`
- [#&#8203;15338](https://togithub.com/babel/babel/pull/15338) Allow
negative number in ambient const initializer
([@&#8203;JLHwung](https://togithub.com/JLHwung))

##### ↩️ Revert

- `babel-plugin-transform-react-inline-elements`,
`babel-plugin-transform-react-jsx-development`,
`babel-plugin-transform-react-jsx`
- [#&#8203;15355](https://togithub.com/babel/babel/pull/15355) Reverts
"Re-use common JSX element transform for <>...\</>"
([@&#8203;JLHwung](https://togithub.com/JLHwung))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "every weekday" (UTC), Automerge - At
any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://app.renovatebot.com/dashboard#github/TryGhost/Ghost).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNC4xMDguNCIsInVwZGF0ZWRJblZlciI6IjM0LjE0Ni4xIn0=-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-03-01 15:44:09 +00:00
Sanne de Vries
2c34d0841c Updated copy for "Click to update" banner
Refs https://github.com/TryGhost/Team/issues/2400
2023-03-01 15:41:13 +01:00
Simon Backx
81c4b46977
Grouped mentions from the same source (#16348)
fixes https://github.com/TryGhost/Team/issues/2625

- Adds an unique option to the mentions API. Enabling this will only
return the latest mention from each source.
- The frontend can fetch the related sources for each page by doing an
extra request to the mentions API.
2023-03-01 12:15:29 +01:00
Daniel Lockyer
22a2f194aa
Removed deprecated webpack config
- we no longer use this because we configure webpack via
  `scripts/build-combined.js`
2023-03-01 11:51:51 +01:00
Naz
eb8d9df923
Made image importer type dynamic
refs https://github.com/TryGhost/Toolbox/issues/523

- Having a dynamic importer type allows to reuse the image importer implementation for other file types e.g.: 'media'
2023-03-01 18:28:17 +08:00
Naz
edebe1571c
Extracted image store to constructor parameter
refs https://github.com/TryGhost/Toolbox/issues/523

- We need to be able to use different storage mechanisms when importing different types of content
- Having the storage passed in using constructor DI allows to have more flexible storage mechanism in the Images importer (soon to become a generic file importer)
2023-03-01 18:21:46 +08:00
Naz
4573d8b4b8
Refactored image importer to use class
refs https://github.com/TryGhost/Toolbox/issues/523

- The class syntax would allow swapping out the storage mechanism in the importer making it universal to use with other file types like media or generic files.
2023-03-01 18:10:39 +08:00
Naz
2bf98e288f
Removed unused module from test suite
refs https://github.com/TryGhost/Toolbox/issues/523
2023-03-01 18:10:39 +08:00
Daniel Lockyer
f717c093e7
Fixed linting 2023-03-01 10:30:18 +01:00
Daniel Lockyer
6d8ca23625
Added support for namespaces to i18n package
refs https://github.com/TryGhost/Ghost/issues/15502

- this adds support for namespaces to the i18n package so we can
  separate translations for different packages
2023-03-01 10:24:08 +01:00
Sanne de Vries
150a3a9c68 Added "Click to update" banner to admin
Refs https://github.com/TryGhost/Team/issues/2400
- Adds a banner to the admin to indicate that a new version is available
- This is just the UI that hasn't been wired up to the actual version check yet
2023-03-01 10:05:23 +01:00
Ronald Langeveld
4407e8e49e
🐛 Added undefined error handling to failed uploads (#15982)
fixes https://github.com/TryGhost/Team/issues/2320

- Adds error handling when undefined objects are passed to certain upload
functions such as `videoUploadCompleted`.
2023-03-01 16:50:56 +08:00
Naz
e14520115a
Extracted Image Importer test suite
refs https://github.com/TryGhost/Toolbox/issues/523
refs 3119a5cc4c

- The "index.test.js" is doing too much, has to be broken down to allow things to move into new packages
2023-03-01 16:48:38 +08:00
Daniel Lockyer
4c2ff0539d
Fixed tests for ImportManager audio types
refs c5a4ee89c9

- as of the referenced commit, the supported list of mime types has been
  updated
2023-03-01 09:14:09 +01:00
Naz
6c9e046b4b
Reduced urlUtils dependency in ImageHandler
refs https://github.com/TryGhost/Toolbox/issues/523

- url-utils module is actively being stripped down from usages across codebase. The "urlUtils.STATIC_IMAGE_URL_PREFIX" and "store.staticFileURLPrefix" are the same values - the main difference is with the value coming from store we can make this module more generic for any store type (images, media, files, etc)
2023-03-01 16:10:54 +08:00
Naz
3119a5cc4c
Extracted ImageHandler test suite
refs https://github.com/TryGhost/Toolbox/issues/523

- The "importer/index.test.js" test suite is testing more than it should. ImageHandler test suite section is one of the examples of test cases that should live in a separate file.
- Having these tests in different files makes it easier to reason about coverage and extract to it's own packages.
2023-03-01 15:55:10 +08:00
Naz
ce5eea5707
Added media file handling to Ghost importer
refs https://github.com/TryGhost/Toolbox/issues/523

- When a zip file is imported into Ghost we need to recognize and process media files with following extensions:
".mp4",".webm", ".ogv", ".mp3", ".wav", ".ogg", ".m4a"
- The media files can come from a "media" or "content/media" folder inside of zip file
2023-03-01 15:55:10 +08:00
Naz
9347e996f8
Created Media Importer Handler package
refs https://github.com/TryGhost/Toolbox/issues/523

- We need media file imports. Media handler is one of the pre-processing parts that need to be done during the import process
- Media import handler is handling import files with following extensions: ".mp4",".webm", ".ogv", ".mp3", ".wav", ".ogg", ".m4a"
- The implementation is largely a copy-paste with class syntax adjustments from the "/core/server/data/importer/handles/image.js" module
- There are a lot of code similarities between media and image import handlers. The new "ImporterMedia" class could serve as a generic base class for file-storage-related imports
2023-03-01 15:55:10 +08:00
Naz
b768e710f1
Bootstrapped media importer package
refs https://github.com/TryGhost/Toolbox/issues/523

- This is a base for the media importer package
2023-03-01 15:55:10 +08:00
Naz
23e2fdde95
Added image importer test coverage
refs https://github.com/TryGhost/Toolbox/issues/523

- The test cases for "content" and "content/images" zip import file structure were missing.
2023-03-01 15:55:10 +08:00
renovate[bot]
6184bbcceb Update peter-evans/create-or-update-comment digest to b1fa5d5 2023-03-01 08:45:32 +01:00
renovate[bot]
a431c16968 Update dependency cli-progress to v3.12.0 2023-03-01 08:45:10 +01:00
monkey sees
838516efb0 🐛 Fixed Handlebars’ asset helper for SafeString input
closes https://github.com/TryGhost/Ghost/issues/16332

Passing `SafeString` input to `asset` helper was resulting in the
exception being thrown. This meant that we couldn’t combine `asset`
helper with other helpers which produce `SafeString` e.g. `concat`
helper for string concatenation.
2023-03-01 08:39:14 +01:00
Reupen Shah
c5a4ee89c9 🐛 Fixed uploads of m4a files with audio/mp4 content type
fixes https://github.com/TryGhost/Ghost/issues/16301

Previously, audio/x-m4a was allowed but not audio/mp4. This meant
uploads of m4a files failed in some cases e.g. Firefox on Windows.
2023-03-01 08:32:21 +01:00
Djordje Vlaisavljevic
d1657b90ea Added mentions dashboard icons
refs https://github.com/TryGhost/Team/issues/2608
2023-02-28 22:23:06 +00:00
Djordje Vlaisavljevic
2b7c5ca66a Updated recent mentions dashboard widget design
refs https://github.com/TryGhost/Team/issues/2608
2023-02-28 22:23:06 +00:00
renovate[bot]
41e01f6531 Update peter-evans/create-or-update-comment digest to c5a7a98 2023-02-28 19:09:58 +01:00
Daniel Lockyer
d383aeafda
Updated lockfile 2023-02-28 18:29:54 +01:00
Steve Larson
9e58181398
combined fetch for metadata and verification (#16330)
refs TryGhost/Team#2582
-removed duplicative fetch request
-added mention as type in oembed service
2023-02-28 08:39:28 -06:00
Djordje Vlaisavljevic
cd0115deec Added static design for grouped mentions with popover
refs https://github.com/TryGhost/Team/issues/2606
2023-02-28 13:37:34 +00:00
Kevin Ansfield
26d05aecd8
Fixed Theme.errors clash with Ember Data Model's errors property (#16106)
no issue

- the Ember Data `Model` class has an `errors` property by default that
is set to a `DS.Errors` instance but the Theme model was overriding that
with an `errors` attr
- it hasn't been an issue so far but causes problems in Ember/Ember Data
3.28.x because that tries to use the `DS.Errors` interface on the
overridden attr property which then throws errors because the `errors`
attr doesn't have the right methods
2023-02-28 13:28:32 +00:00