Commit Graph

19027 Commits

Author SHA1 Message Date
Naz
5f2967cf27 Added support for offloaded oneoff jobs
refs https://github.com/TryGhost/Toolbox/issues/357

- Adds support for persisted one off offloaded (worker thread) jobs
- To try them out run Ghost instance in "testmode" and shoo a request like so: `curl http://localhost:2368/ghost/api/oneoff/graceful-job` - this starts a one time job from graceful-job script (can only ever be executed once on the Ghost instance)
- Job's progress and runtime details are persisted in `jobs` table
- To play more with one off jobs use `addOneOffJob` method available on jobsService
2022-07-22 18:19:07 +01:00
Naz
d5e38e6fc7 Published new versions
- @tryghost/adapter-manager@0.2.33
 - @tryghost/api-version-compatibility-service@0.4.4
 - @tryghost/bootstrap-socket@0.2.22
 - @tryghost/config-url-helpers@1.0.2
 - @tryghost/constants@1.0.7
 - @tryghost/database-info@0.3.8
 - @tryghost/email-content-generator@0.1.4
 - @tryghost/image-transform@1.2.1
 - @tryghost/job-manager@0.9.0
 - @tryghost/limit-service@1.2.2
 - @tryghost/minifier@0.1.17
 - @tryghost/moleculer-service-from-class@0.2.28
 - @tryghost/mw-api-version-mismatch@0.2.3
 - @tryghost/mw-error-handler@1.0.5
 - @tryghost/mw-session-from-token@0.1.34
 - @tryghost/mw-update-user-last-seen@0.1.8
 - @tryghost/package-json@1.0.23
 - @tryghost/pretty-cli@1.2.29
 - @tryghost/promise@0.1.20
 - @tryghost/release-utils@0.8.1
 - @tryghost/security@0.3.3
 - @tryghost/session-service@0.1.44
 - @tryghost/settings-path-manager@0.1.9
 - @tryghost/version-notifications-data-service@0.2.2
 - @tryghost/vhost-middleware@1.0.28
 - @tryghost/zip@1.1.27
2022-07-22 16:46:56 +01:00
Naz
df8a7f9b6b Fixed formatting 2022-07-22 16:44:28 +01:00
Naz
b0581c778e Added execution progress updates for one off jobs
refs https://github.com/TryGhost/Toolbox/issues/357

- Job persisted in the database need to track job's execution status such as completion, failure, execution start and end times. This changeset allows to hook into job/bree lifecycle to track job's progress.
- NOTE: only supports "offloaded" jobs at the moment. Support for "inline" jobs will be added once there's a clear usecase for it.
- The "started" status and "started_at" timestamp are assigned to a job at the moment when the worker thread is created inside of bree
- The "finished" status and "finished_at" timestamp are assigned to a job when a "done" event is passed from the job script (NOTE: using process.exit(0) will not trigger the "finished" state")
- The "failed" status is assigned when the job execution is interrupted with an error
2022-07-22 16:43:15 +01:00
renovate[bot]
c9ae36ea8c Update dependency date-fns to v2.29.1 2022-07-22 13:04:15 +00:00
renovate[bot]
e9132d7572
Update dependency sqlite3 to v5.0.10 2022-07-22 12:09:33 +00:00
Naz
c667620d8f 🏗 Added jobs table creation migration
refs https://github.com/TryGhost/Toolbox/issues/357

- One time jobs need a storage mechanism to be run only ever once.
- Field notes:

- `id`, `created_at`, `updated_at` - standard Ghost fields
- `name` - unique name of the job, could also be used with prefixing to identify certain type of job (e.g.: backup-bob-2022-10-16, backup-sam-2023-01-13 identifying backup jobs run by users)
- `status` - 'started' | 'finished' | 'failed' | 'queued'  (need to identify when the job is in progress, done, added to the execution queue, or errored)
- `started_at` - when the job started execution
- `finished_at` - when the job successfully finished execution
2022-07-22 23:32:58 +12:00
Daniel Lockyer
cff4772d50
Removed Members packages from auto-assign rules
- there won't be any package bumps matching this now they are merged
  into this monorepo
2022-07-22 12:40:29 +02:00
Daniel Lockyer
fc1f651db6
Fixed disabling of Ghost-CLI tests
- conditional binding rules are difficult so let's just make it simpler
2022-07-22 12:20:40 +02:00
Simon Backx
30c4f11e27
Added report API for comments (#15043)
closes https://github.com/TryGhost/Team/issues/1684

**Migrations:**
- Added report permissions (fixtures + migrations)
- Dropped reason field in reports (no textarea in reports in V1)
- Dropped nullable from comment_likes.member_id (can't be null)
- Added SET NULL/CASCADE foreign keys for comments related tables(*)

(*):
fixes https://github.com/TryGhost/Team/issues/1687
refs https://ghost.slack.com/archives/C02G9E68C/p1658217288591369

This commit adds support for `SET NULL` foreign keys in schema and migration helpers + also fixes the foreign keys for the comment_reports, comment_likes and comments tables.

- When a member is deleted, we **do** want to keep their reports (SET NULL)
- When a member is deleted, we **do not** want to keep their likes (CASCADE)
- When a member is deleted, we **do** want to keep the comments (SET NULL)

**Changes:**
- Added report API: `POST /members/api/comments/{id}}/report/`
- Sends an email to the owner when a comment is reported
- Saves a report to the database (not used for now, but might be useful later)
2022-07-22 12:03:05 +02:00
Daniel Lockyer
c6e4127eaa
Removed ghost-cli dependency for canary build
- I had to disable ghost-cli tests whilst we're converting to a
  monorepo, so the canary build isn't triggered
2022-07-22 10:36:47 +02:00
renovate[bot]
2339577dd3 Update dependency metascraper-logo-favicon to v5.30.0 2022-07-22 08:34:07 +00:00
renovate[bot]
33b3fab663 Update dependency @playwright/test to v1.24.0 2022-07-22 09:39:38 +02:00
Naz
5dae6d6acf Added support for one off jobs
refs https://github.com/TryGhost/Toolbox/issues/357

- This is a scaffolding for what will become a one off job scheduling mechanism. The aim is allowing to run jobs which can be only ever be run once in the lifetime of the instance - persisting through restarts.
2022-07-21 19:39:54 +01:00
renovate[bot]
c34edc9f26
Pin dependency pump to 3.0.0 2022-07-21 18:04:31 +00:00
Naz
13bfc0746b Revert "Transformed addJob to be an async method"
This reverts commit f8b1877253.
2022-07-21 18:56:35 +01:00
Naz
3e57e6fe10 Moved jobmanager initialization logic
refs https://github.com/TryGhost/Toolbox/issues/357

- In upcoming iteration job manager will need "models" fully initialized to be able to persiste one time jobs. This bit of code of in a bad place as it initialized (through require) the job manager without having models module initialized first.
- The change moves that code from a bad to less bad place (wasn't able to think of any better location for now). Checking for `server:testmode' config in job initialization has a little bit of a smell, but don't think it's super critical. Could be improved if a better structure appears in the future!
2022-07-21 17:00:45 +01:00
Daniel Lockyer
17bf358b47
Pinned internal packages
- we're going to be pinning all dependencies within the monorepo
- this shouldn't change anything anyway because we're using the same
  version across all packages
2022-07-21 16:35:19 +02:00
renovate[bot]
d228144c27 Pin dependencies 2022-07-21 16:27:57 +02:00
renovate[bot]
6ebe34fdf0 Update dependency @playwright/test to v1.23.4 2022-07-21 14:37:19 +02:00
Daniel Lockyer
3559c57dbe
Fixed package import
refs 2c59f5342b

- this was missed during the referenced commit
2022-07-21 12:41:54 +02:00
Daniel Lockyer
2c59f5342b
Switched away from using @nexes packages
- we renamed the scope to `@tryghost` a while back, and published new
  versions, but these packages seem to have been missed during the
  rename
2022-07-21 12:38:09 +02:00
Daniel Lockyer
ac3d4d5085
Restored yarn main
- still a bit of a hack right now but it should pull down the latest
  changes
2022-07-21 10:11:54 +02:00
Daniel Lockyer
48fd5ca8cb
Removed posttest linting step
- linting is done as a separate step in CI and it's a git hook upon
  pushing locally, so we don't need to run it after tests
2022-07-21 10:09:36 +02:00
Daniel Lockyer
03cb00e7fb
Unified some package scripts
- reworked lint script to follow the format of others
- deleted posttest linting from members-ssr
2022-07-21 09:51:16 +02:00
Daniel Lockyer
5940fa630b
Fixed linting and unit tests running on all packages
- this was leftover from the transition to a monorepo but we should be
  running linting and tests on all packages
2022-07-21 09:44:45 +02:00
Daniel Lockyer
6192ae4620
Increased E2E test timeout temporarily
- we keep seeing a lot of random failures from CI due to the Posts
  Content API
- I think it's because of the Ghost boot + number of fixtures we load,
  which sends us over our 10s threshold
- let's see if 15s resolves that
2022-07-21 09:40:09 +02:00
Daniel Lockyer
d5874e29a3
Removed members-api posttest linting step
- we don't need to run linting after testing, as this is done separately
  and upon a `git push` anyway
2022-07-21 09:30:07 +02:00
Daniel Lockyer
9ec83a6e21
Disabled publishing for Members packages
- we don't want to publish these anymore so this commit disables the
  ability to
- also fixes up a missing version that wasn't reset
2022-07-21 09:26:04 +02:00
Daniel Lockyer
ed7e17b2f9
Deleted commited lockfile
- this was leftover from the Members package import
2022-07-21 09:19:39 +02:00
Daniel Lockyer
376ee24600
Switched to unversioned Members packages
- these packages are split apart for local development, but will be
  bundled into Ghost when publishing
- therefore, these packages won't be published so we are resetting the
  versions to make them cleaner
2022-07-21 09:15:29 +02:00
Daniel Lockyer
ec7dad9a52
Updated lockfile 2022-07-21 09:06:30 +02:00
Daniel Lockyer
67aebbc7bc
Migrated Members packages into Ghost monorepo
refs https://github.com/TryGhost/Toolbox/issues/354

- now the Ghost repo is a monorepo, we can migrate our packages in to
  make development easier
2022-07-21 09:04:30 +02:00
Naz
f8b1877253 Transformed addJob to be an async method
https://github.com/TryGhost/Toolbox/issues/357

- This is a groundwork before adding one-off (solo) jobs with persistance to the job manager
- Making the addJob method async also makes the whole interface consistent - removeJob and shutdown are also async
2022-07-20 20:41:02 +01:00
renovate[bot]
6a18b50b8a
Update dependency sanitize-html to v2.7.1 2022-07-20 19:40:39 +00:00
renovate[bot]
16d3045997 Update dependency @types/node to v16.11.45 2022-07-20 18:08:14 +02:00
renovate[bot]
206df11f27 Update dependency @types/node-jose to v1.1.10 2022-07-20 18:08:02 +02:00
Daniel Lockyer
273d53cf09
Updated testing & linting packages
- these weren't updated in core before the migration, so we ended up
  with two different versions in yarn.lock
2022-07-20 18:05:27 +02:00
Daniel Lockyer
08779bc977
Fixed version of email-analytics-service within email-analytics-provider-mailgun
- this was missed during the migration but needs to happen so the
  packages are linked and re-used
2022-07-20 18:00:17 +02:00
Daniel Lockyer
abdcd9a650
Switched to @tryghost/nql-lang
- we moved this from `@nexes` to the `@tryghost` organization a while
  back, and this was missed from the switchover
2022-07-20 17:54:13 +02:00
Daniel Lockyer
ed1bdac20c
Removed publicConfig from packages config
- this was leftover from the migration to the monorepo and won't be
  needed because we're not publishing the packages
2022-07-20 17:49:52 +02:00
Daniel Lockyer
d3930b50ef
Reworked imports for custom-theme-settings-service tests
- now they import from the package exports, we get a slight boost in
  coverage because `index.js` is now used
2022-07-20 17:47:23 +02:00
Daniel Lockyer
e0a12166fc
Enabled unit testing for all packages in CI
- again, a slight hack but this enables us to run tests for all packages
  in CI right now
2022-07-20 17:43:55 +02:00
Daniel Lockyer
796961329a
Fixed yarn test from top-level repo
- we probably don't want it to be this long term but it allows us to
  enable tests on the new packages
2022-07-20 17:41:11 +02:00
Daniel Lockyer
b65816f421
Disabled Ghost-CLI test
- this is broken for now until I have dependency bundling working for
  the monorepo
2022-07-20 17:30:48 +02:00
Daniel Lockyer
9fa789159c
Reset Publishing packages version and visibility
refs https://github.com/TryGhost/Toolbox/issues/354

- these packages are here for development and will be bundled when
  published, so they don't need versioning nor publishing
2022-07-20 17:20:24 +02:00
Daniel Lockyer
685b270013
Deleted .gitkeep file
- this was leftover from the Publishing packages migration
2022-07-20 17:19:04 +02:00
Daniel Lockyer
75d8a29642
Migrated Publishing packages into Ghost repo
refs https://github.com/TryGhost/Toolbox/issues/354

- now we've turned the Ghost repo into a monorepo, we can migrate
  packages back in to make development easier
2022-07-20 17:18:08 +02:00
Daniel Lockyer
70110a39cc
Deleted Release GitHub Action
- I don't want it accidentally triggering whilst I'm importing packages,
  and it's likely to change anyway in the near future
2022-07-20 17:16:07 +02:00
Daniel Lockyer
6f1e7fc40f
Fixed migration review action target paths
- now we've moved to a monorepo, the paths have changed
2022-07-20 16:50:33 +02:00