Commit Graph

32737 Commits

Author SHA1 Message Date
Sam Lord
10a4a8eb12 Fixed tests for demo-data
no issue
2023-02-17 15:08:01 +00:00
Sam Lord
ca395a958c Added post analytics to data generator
no issue
2023-02-17 14:03:59 +00:00
Simon Backx
ffc0cc020a Updated coverage excemption list
Removed the wrapper class for the email service from coverage, because this only wires up a lot of dependencies, which is hard to test in a unit test because we also have to init all the dependencies in a unit test. It is already covered by E2E tests.
2023-02-17 14:27:59 +01:00
Simon Backx
ec3f262ea4 Increased scheduling default test timeout 2023-02-17 13:29:07 +01:00
Fabien "egg" O'Carroll
8908a51547 Improved verification logic for Mentions
refs https://github.com/TryGhost/Team/issues/2550

By using cheerio to parse the HTML we can correctly look for elements
which use the target URL as the href attribute, rather than doing a
plaintext search. This closer to what the spec says.
2023-02-17 19:07:11 +07:00
Fabien "egg" O'Carroll
e14d2e662b Wired up verification to MentionsAPI
refs https://github.com/TryGhost/Team/issues/2550

Whilst this isn't ideal making multiple requests for the same site
it's a first step towards verification properties, and can be
refactored to improve performance later. With the current volume of
incoming Webmentions we've seen so far this shouldn't be a problem.
2023-02-17 19:07:11 +07:00
Fabien "egg" O'Carroll
37cfb96d9f Added support for verified property to BookshelfMentionRepository
refs https://github.com/TryGhost/Team/issues/2549

This ensures we can persist the verification of a Mention to MySQL/SQLite3
2023-02-17 19:07:11 +07:00
Fabien "egg" O'Carroll
277a563138 Added ability to set verified when instantiating Mention
refs https://github.com/TryGhost/Team/issues/2548

We need to be able to do this outside of the verify method so that
repository implementations have the ability to create existing
instances without running expensive verification checks!
2023-02-17 19:07:11 +07:00
Fabien 'egg' O'Carroll
101cabb188
Added separate job service for webmentions (#16285)
[Added initial mentions-jobs
service](3656190114)

This is the result of running `cp -r jobs mentions-jobs` in the services
directory.

[Waited for mentions-jobs queue before
shutdown](2bb1a12a89)

This matches the functionality of the existing jobs service where we
will wait
for jobs to complete before closing the process.

[Used mentions-jobs service in the mentions
service](4e4f9fdd00)

This ensures that any delays in the mentions jobs queue does not effect
other
parts of the application.
2023-02-17 13:05:36 +01:00
Daniel Lockyer
2edfd8b8e0
Increased timeout to allow for delays in browser e2e test
- this test was randomly timing out, and the tests above allow for a
  greater timeout before failing
2023-02-17 12:13:09 +01:00
Aileen Booker
2f57e95a5d
Slack notifications service for Milestones behind flag (#16281)
refs
https://www.notion.so/ghost/Marketing-Milestone-email-campaigns-1d2c9dee3cfa4029863edb16092ad5c4?pvs=4

- Added a `slack-notifications` repository which handles sending Slack
messages to a URL as defined in our Ghost(Pro) config (also includes a
global switch to disable the feature if needed) and listens to
`MilestoneCreatedEvents`.
- Added a `slack-notification` service which listens to the events on
boot.
- In order to have access to further information such as the reason why
a Milestone email hasn't been sent, or the current ARR or Member value
as comparison to the achieved milestone, I added a `meta` object to the
`MilestoneCreatedEvent` which then gets accessible by the event
subscriber. This avoid doing further requests to the DB as we need to
have this information in relation to the event occurred.

---------

Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
2023-02-17 12:59:18 +02:00
Steve Larson
034a230365 Added alpha feature to demonstrate websockets
refs https://github.com/TryGhost/Team/issues/2561

- added simple socket-io implementation to Ghost server
- added alpha flag for websockets
- added route in admin to test websockets using a simple counter stored in server local memory (refreshes on reboot)
2023-02-17 11:55:15 +01:00
Daniel Lockyer
e52efb31b8
Updated coverage exemption list
- like the members jobs, we don't need to run unit tests on these
  because they're not library code
2023-02-17 11:43:13 +01:00
Steve Larson
1e00bd2af9 Handled empty content when sending webmentions
refs https://ghost.slack.com/archives/C02G9E68C/p1676564978732119

- `cheerio` errors when trying to parse `null`
2023-02-17 10:46:48 +01:00
Naz
f27c7ba044
Fixed variable name typo 2023-02-17 17:27:51 +08:00
Naz
3ddfe59e15
Added resource caching to Posts/Pages Content API
refs https://github.com/TryGhost/Toolbox/issues/522

- Browse endpoints for Posts and Pages are creating the most database traffic in the system. These are read-only endpoints that don't have to be fresh 100% of the time. Having optional cache allows to offload some of the database querying to more efficient storage.
- To enable cache for Posts/Pages browse endpoints there are two prerequisites:
  - set 'hostSettings:postsPublicCache:enabled' to 'true' in the configuration file
  - add 'postsPublic' cache adapter in cache configuration

- Example config for adapters with 60s TTL for a cached resource:
```
    "adapters": {
        "cache": {
            "postsPublic": {
                "adapter": "Redis",
                "ttl": 60,
                "keyPrefix": "site_id_here:posts-content-api:"
            }
        }
    },
```
2023-02-17 17:27:51 +08:00
Naz
c3d7104367
Refactored tags repository to be model-agnostic
refs https://github.com/TryGhost/Toolbox/issues/522

- The caching rules for Content API resources like Tags / Posts / Pages / Authors are the same, so it makes sense to make a common repository which would take in a model as a parameter and perform the queries on that level instead of implementing repository-per-model.
- With this refactor getting Posts Content API caching would be much simpler change.
2023-02-17 17:27:51 +08:00
Steve Larson
5d00b64e27
Added timeout to external requests
no refs

- got doesn't have a timeout by default
- we think we might have hanging jobs because of this
2023-02-17 09:16:58 +01:00
Daniel Lockyer
9a2d765c73
Added one-off job name and current queue length into log line
- this makes it easier to know what jobs are being scheduled and what
  the current queue length is
2023-02-17 09:16:40 +01:00
Aileen Nowak
cd3930d08e Promote milestoneEmails to beta feature
no issue

- Promotes `milestoneEmails` to a beta feature so it can be a private beta to be tested
2023-02-17 07:45:51 +02:00
Aileen Nowak
1c0ee675fe Updated milestone values in default config 2023-02-17 07:45:51 +02:00
renovate[bot]
7de35f1ca3 Update dependency node-jose to v2.2.0 [SECURITY] 2023-02-17 05:29:46 +00:00
renovate[bot]
cf7ecb492c
Update dependency c8 to v7.13.0 2023-02-16 22:15:50 +00:00
Sam Lord
b4c90bf07d Reduced number of login events in large-scale data generation
no issue

This should massively increase the speed of importing for the large dataset, which is important as the time to import it on Pro is >10 minutes at the moment
2023-02-16 20:54:14 +00:00
renovate[bot]
9bd2b8c708 Update dependency lint-staged to v13.1.2 2023-02-16 15:01:58 +01:00
renovate[bot]
3e4cb3f85e Update peter-evans/create-or-update-comment digest to 300bfad 2023-02-16 15:01:36 +01:00
renovate[bot]
38642af435 Update dependency cssnano to v5.1.15 2023-02-16 14:59:59 +01:00
Sam Lord
b8f59f07f2 Added --clear-database option to make it easier to use the data generator
no issue
2023-02-16 12:11:08 +00:00
Simon Backx
f8e77ad012 🐛 Fixed uploading images with custom storage adapters
fixes https://github.com/TryGhost/Ghost/issues/16278

`urlToPath` is not a method that is defined on the StorageBase, and thus is not implemented on any custom storage adapter.

We need this method to prevent uploading a file in two different directories. Currently this is an edge case:
- If you upload a file at midnight between a month shift, it is possible that we upload one file at `/2023/02/file.png` and the original at `/2023/03/file_o.png`
- To make sure we upload the second at the same directory, we need the `urlToPath` implementation. The save method sadly only returns the URL, which could be different than the stored file path, so we cannot use that return value.

This fix only uses `urlToPath` if available, to prevent above issue.
2023-02-16 11:49:37 +01:00
Simon Backx
8dfa490638
Added outbound link tagging for web posts (#16201)
fixes https://github.com/TryGhost/Team/issues/2433

- Moved all outbound link tagging code to separate OutboundLinkTagger
- Because a site can easily enable/disable this feature, we don't store
the ?refs in the HTML but add them on the fly for now in the Content
API.
2023-02-16 11:26:35 +01:00
Fabien "egg" O'Carroll
a310ec2e3e Fixed Mention unit test
refs https://github.com/TryGhost/Ghost/commit/c6b90b6ac
2023-02-16 14:37:22 +07:00
Fabien "egg" O'Carroll
c6b90b6ac7 Added verified property to Mentions Admin API
closes https://github.com/TryGhost/Team/issues/2551

Rather than blindly passing all data through the API we explicitly include each
new property. This allows us to make changes to the core entities without
affecting the API. The verified property is being added now to give design the
ability to display these mentions differently.

We also needed to include the verified property in the return value of toJSON,
this was missed as part of the original entity changes
2023-02-16 14:31:05 +07:00
Ghost CI
dd48f65712 Merged v5.34.1 into main 2023-02-16 07:23:52 +00:00
Ghost CI
443085ab87 v5.34.1 2023-02-16 07:23:49 +00:00
Naz
8a0c20b54c
Fixed Redis cluster ttl configuration
refs https://github.com/TryGhost/Toolbox/issues/520

- The cluster config is taking over local adapter ttl configuration - the priority should be reverse adapter config first followed by cluster config
- In addition if we add nested config merging to adapter manager we could achieve the same effect by having per-adapter configuraiton with "clusterConfig.options.ttl" value
2023-02-16 14:13:44 +08:00
John O'Nolan
14c5e48190
Improved micro copy on Portal email tips 2023-02-15 12:30:03 -04:00
Aileen Nowak
5f12c7ad40 Updated Milstone test to expect no event for existing milestone 2023-02-15 14:28:57 +02:00
Aileen Nowak
dc8dc90212 Exposed MilestoneCreatedEvent as module 2023-02-15 14:28:57 +02:00
Aileen Nowak
d482046a25 Moved event creation to all save events and not only new ones 2023-02-15 14:28:57 +02:00
Aileen Nowak
b55a95b944 Removed usage of GhostMailer in MilestonesService
no issue

- With the switch of using a `MilestoneCreatedEvent` we'll be decoupling the mailing functionality and not need `GhostMailer` as dependency in the package anymore
2023-02-15 14:28:57 +02:00
Aileen Nowak
513b7d1df4 Added MilestoneCreatedEvent using DomainEvents
no issue

- In preparation of using event emitting for Milestone achievements, we needed to add a dedicated `MilestoneCreatedEvent` to the `Milestone` entity.
- The event will be emitted using `DomainEvents` when a new milesteone is saved, which will allow us to listen to these events.
2023-02-15 14:28:57 +02:00
renovate[bot]
636ff6d4ef Update sentry-javascript monorepo to v7.37.2 2023-02-15 11:25:44 +01:00
Aileen Nowak
a400353881 Renamed milestone-emails to milestones
no issue

- The way we're going to implement milestones diverged from the original idea of handling email sending within the milestone-emails package, as we'll be sending events instead and will utilise the StaffService to listen to them and send the emails
- This renames the package as well as the service in core itself and all relevant tests
2023-02-15 10:43:05 +02:00
Aileen Booker
41e4132a17
Refactored milestones service to preserve API instance (#16272)
no issue

- For better testability with in-memory repository, refactor the
milestones service to preserve the API instance
- Fetching the information about Stripe live mode from Stripe service
was causing difficulties when testing. As a workaround we switched to
reading the live mode keys and determine it that way.

---------

Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
2023-02-15 08:43:40 +02:00
Naz
d62eba6c7f
Fixed Redis cluster ttl configuration
refs https://github.com/TryGhost/Toolbox/issues/520

- The cluster config is taking over local adapter ttl configuration - the priority should be reverse adapter config first followed by cluster config
- In addition if we add nested config merging to adapter manager we could achieve the same effect by having per-adapter configuraiton with "clusterConfig.options.ttl" value
2023-02-15 11:39:04 +08:00
Ronald Langeveld
f9eeec10db Updated email-preview test snapshot
no issue

- fixed up tests failing from the outdated email snapshot.
2023-02-15 11:21:37 +08:00
Peter Zimon
5bb5977a47 Fixed issue with bookmark cards width on Android
no refs.

- On Android Gmail client the bookmark cards were not taking the full width of the screen
2023-02-14 14:31:16 +01:00
Ronald Langeveld
c651f5b10f
Added verified mention column (#16264)
refs https://github.com/TryGhost/Team/issues/2549

- Added `verified` column to mentions table
- Added a new migration
- Updated schema
- Updated integrity test
2023-02-14 15:46:07 +08:00
renovate[bot]
7f278bd762 Update Test & linting packages 2023-02-14 04:25:29 +00:00
John O'Nolan
0334595a0e Updated post preview meta description prompt 2023-02-13 16:58:28 -04:00