Commit Graph

98 Commits

Author SHA1 Message Date
Daniel Lockyer
6dc1d08590 Re-enabled general eslint rules in TS config
refs https://github.com/TryGhost/DevOps/issues/50

- when creating a TS config in our `eslint-plugin-ghost` dependency, I
  only extended the recommended config, which left out a lot of
  stylistic things we used to enforce in JS
- this fixes that by bumping the dependency to a version which extends
  those shared configs, and fixes all the code that currently goes
  against those rules
2023-09-08 13:47:42 +02:00
Fabien "egg" O'Carroll
c9fff97c75 fixup! Added logging to collections event handlers 2023-09-08 14:46:38 +07:00
Fabien "egg" O'Carroll
ff8a3fbf32 fixup! Added logging to collections event handlers 2023-09-08 14:46:38 +07:00
Fabien "egg" O'Carroll
c6f908d31d Added logging to collections event handlers
This will help us with debugging issues we're seeing with large sites
2023-09-08 14:46:38 +07:00
Naz
fc5f139d39 Removed stray console.log
refs 367584f1c2

- I fucked up. ESLint fucked up. Life happened to all of us
2023-09-08 15:28:47 +08:00
Naz
367584f1c2 Optimized PostEditedEvent processing in collections
refs https://github.com/TryGhost/Arch/issues/86

- When PostEditedEvent data contains no visible changes we can skip the matching collections update process alltogether. Each call to `updatePostInMatchingCollections` creates a transaction in addition to fetching all collections. There's no need to process anything when there are no relevant changes in the post edit!
2023-09-08 13:46:44 +08:00
Daniel Lockyer
c6cb35074a Updated linting and testing packages 2023-09-01 15:51:17 +02:00
Naz
608d4dfa10 Optimized the amount of logs produced by collections module
refs https://github.com/TryGhost/Arch/issues/82

- Collections logs are too verbose causing noise.
- Moved some of the logs to use "debug" for now and made summarized logs for the information that we still need while collections code is actively monitored. The event info logs can be removed once we are passed the active phase of rolling out the collections feature
2023-09-01 12:40:54 +08:00
Naz
acccc16614 Added collections update after bulk adding tags
refs https://github.com/TryGhost/Arch/issues/77

- During initial development we have missed to support collections update when tags are added to posts in bulk. It's especially valid usecase since we can define automatic collection with a filter containing not yet existing tags.
2023-08-23 17:12:08 +08:00
Fabien "egg" O'Carroll
62d5ca558d Validated filters for collections
refs https://github.com/TryGhost/Arch/issues/47

This ensures that we only have collections which have a valid filter in terms of
  - Valid NQL string
  - Uses only properties which are valid to filter on
  - Only has an empty filter in the case of the "latest" collection
2023-08-23 15:49:06 +07:00
Fabien "egg" O'Carroll
ad62cd4822 Removed edit method from Collection
We want to keep Entities as basic as possible, particularly when setting
property, so that each property can handle its own validation.
2023-08-17 09:45:41 +01:00
Naz
aaf0998129 Fixed transaction completion error on collection update
refs https://github.com/TryGhost/Arch/issues/16

- Without an extra await in the update function the passed in transaction would complete before all updates had a chance to run within this transaction.
2023-08-01 15:11:00 +08:00
Naz
fe4c0b18cf Removed unnecessary getBulk in PostsRepository
refs https://github.com/TryGhost/Arch/issues/16

- The method ended up being used only once, so it makes no sense to complicate the interface without any gain.
2023-08-01 15:11:00 +08:00
Naz
8635f4efeb Added collections handling for PostsBulkFeaturedEvent/PostsBulkUnfeaturedEvent
refs https://github.com/TryGhost/Arch/issues/16

- When posts produce PostsBulkFeaturedEvent/PostsBulkUnfeaturedEvent the collections having a featured filter should update the posts belonging to them.
2023-08-01 15:11:00 +08:00
Naz
8046c33194 Added collections handling for PostsBulkUnpublishedEvent
refs https://github.com/TryGhost/Arch/issues/16

- When posts produce PostsBulkUnpublishedEvent the collections having a published_at filter should update the posts belonging to them
2023-08-01 15:11:00 +08:00
Naz
a44d4a24d3 Fixed "any" types for Knex transaction objects
no issue

- We should use specific types wherever possible instead of "any" as it's an antipattern that spreads like cancer through the codebase.
2023-08-01 15:11:00 +08:00
Naz
f4143a8939 Fixed build scripts for post-events package 2023-08-01 15:11:00 +08:00
Naz
9b257f3966 Update of collection posts on bulk destroy
refs https://github.com/TryGhost/Arch/issues/16

- When the bulk destroy is done on posts Collections need to know about the update and remove the stored posts from collections.
2023-08-01 15:11:00 +08:00
Daniel Lockyer
6dd18d81d4 Enabled no-explicit-any for majority of packages
refs https://github.com/TryGhost/DevOps/issues/50

- we should default to keeping the rule on and so I've excluded lines
  that currently use `any` to avoid the need to go and fix them all up
2023-07-27 16:49:08 +02:00
Fabien "egg" O'Carroll
fa40485fb1 Handled updating Collections on TagDeletedEvent
closes https://github.com/TryGhost/Arch/issues/62

Because there are many ways in which filters can rely on tags, we will just
recalculate all automatic collections for now, rather than attempting to do
optimised updates.
2023-07-27 16:33:16 +02:00
Fabien "egg" O'Carroll
9288f56649 Improved type definitions
The PostRepository type was using `any` (an anti pattern) rather than
`PostCollection`, and we had optional properties, which are not really
optional. This cleans up the types and updates the tests alongside them.
2023-07-27 16:33:16 +02:00
Daniel Lockyer
c97cc08455 Updated ESLint config for React+Typescript packages
refs https://github.com/TryGhost/DevOps/issues/50

- `react-app` comes from `eslint-config-react-app`, which is a CRA package
- we're moving away from that so this commit switches the linting over
  to a more recently updated plugin
- once that was removed, we started using a newer version of
  `@typescript-eslint/eslint-plugin`, so there were plenty of
  updates/exemptions to make
2023-07-27 11:40:31 +02:00
Fabien "egg" O'Carroll
607ea8dcd7 Added TagDeletedEvent
refs https://github.com/TryGhost/Arch/issues/60

This will be used to update collections when a tag is deleted. Like the Post
events this should not be in the collections package, instead we should have
these as part of the tags and posts packages. These packages don't exist right
now, so I'm following the existing pattern.
2023-07-26 12:54:42 +02:00
Daniel Lockyer
b0cf1f949a Switched TypeScript eslint config to custom plugin
refs https://github.com/TryGhost/DevOps/issues/50

- this switches the .eslint configs from `node` to `ts`, which is a new
  config to support eslint for TypeScript
- also makes minor changes to adhere to these new rules
2023-07-25 16:12:35 +02:00
Naz
694ab1d32d Added support for expansions in collection filters
refs https://github.com/TryGhost/Arch/issues/46

- Similarly to post filters, collection filters now support both 'tag' and 'tags' nql filter keys when defining a filter for related tag slugs. For example, both `tag:avocado` and `tags:avocado` would both be valid collection filters that would filter by the same 'slug' property of the tags assigned to a post.
- Along with these changes had to rework the tags property of the collection posts to match the shape used in post resources. Moved from:
`tags: ['bacon', 'broc']`
to
`tags:[{slug: 'bacon'}, {slug: 'broc'}]`
2023-07-25 19:01:56 +08:00
Daniel Lockyer
d5cbb33e54 Pinned dependencies 2023-07-24 16:21:47 +02:00
Daniel Lockyer
bb4c3de992 Deduplicated ts-node and typescript dependencies
refs https://github.com/TryGhost/DevOps/issues/50

- this commit deduplicates where we define the versions of these
  packages to make them easier to maintain
2023-07-24 16:21:47 +02:00
Naz
9a38c65ee6 Cleaned up CollectionsService from unused code
refs https://github.com/TryGhost/Arch/issues/16

- Some of the methods became unused due to moving the posts handling out of the collections service and cleaning up the event handling system.
2023-07-21 19:46:22 +08:00
Naz
6361423ff7 Fixed ability to remove post from an automatic collection
refs https://github.com/TryGhost/Arch/issues/16

- Post can be removed from a manual collection, but never from an automatic collection.
2023-07-21 19:46:22 +08:00
Naz
d7bbb0b935 Reworked PostsRepository to use model layer
refs https://github.com/TryGhost/Arch/issues/16

- Using the API directly on the repository level prevented us from ensuring collection consistency through transactions.
- This change migrates the PostsRepository to use Bookshelf model layer directly, which also allows to put queries into transactions.
- Additional optimization here was removing the `getAllPosts` method from CollectionService. This is an attempt to reduce the API surface of the  of the service before calling it a GA.
2023-07-21 19:46:22 +08:00
Naz
3dc27f505c Wrapped collection's creation in transaction
refs https://github.com/TryGhost/Arch/issues/16

- This is needed to avoid stale data/race conditions when processing collection post updates
2023-07-21 19:46:22 +08:00
Naz
0880770d50 Wrapped collection's post adding logic in transaction
refs https://github.com/TryGhost/Arch/issues/16

- Having transactional collection post updates makes sure there are no race conditions when updating collection_posts relations. Without the transactions collection was prone to update relations based on a stale state causing problems like described in the linked issue
2023-07-21 19:46:22 +08:00
Naz
04c92d2ca5 Removed CollectionResourceChangeEvent handling
refs https://github.com/TryGhost/Arch/issues/16

- The generic "UpdateAllCollections" logic should not be used as the mapped Post's delete/add/edit events should be sufficient in maintaining collection's state
2023-07-21 19:46:22 +08:00
Naz
53f9f954c1 Added tag filter support to collections
refs https://github.com/TryGhost/Arch/issues/41

- When an new collection is created the relational "tags" filter is now picked up properly and appropriate posts matching the tag filter are assigned and stored in the collection. Example collection filter that is now supported: `tags:['bacon']`
- Additionally cleaned up returned collection post DTOs, so we return as little data as possible and add only the fields that are needed
2023-07-18 20:18:54 +08:00
Naz
939a8fef33 Moved CollectionPost type to it's own file
refs https://github.com/TryGhost/Arch/issues/41

- This type has to be reused in other class, so best way to do it is having it separated into it's own file
2023-07-18 20:18:54 +08:00
Naz
eec610dc53 Added collection handling for post's tag attach events
refs https://github.com/TryGhost/Arch/issues/41

- When a tag is attached or detached to the post automatic collections matching  the tag filter should be updated.
2023-07-18 18:36:49 +08:00
Naz
c733424cc5 Fixed collections ordering
refs https://github.com/TryGhost/Arch/issues/25

- When run against different DB Engines the returned order of collections belonging to a post is not consistent (SQLite vs MySQL). Having a primitive ordering by slug allows to keep the order compatible
2023-07-17 18:43:21 +08:00
Naz
ed91412cd5 Changed "index" collection name to "latest"
https://github.com/TryGhost/Arch/issues/25

- This naming matches how the collection will be visible in the cards and how it's marketed. Naming it "index" even for internal purposes will get confusing.
2023-07-17 18:43:21 +08:00
Naz
0be06f0237 Added extra logging to collections event processing
no issue

- These logs are useful when tracking event processing by the collections repo.
2023-07-14 14:20:37 +08:00
Daniel Lockyer
2aa7da3a0b Improved monorepo tooling setup
refs https://github.com/TryGhost/DevOps/issues/45

- this switches the monorepo over to using Nx instead of Lerna, because
  we don't currently need the versioning+publishing capabilities
- this also adds an `nx.json`, which allows us to enable task caching
- also adds `build:ts` to the TS projects, which is cached for fast execution
- how these interact with the dev.js script will hopefully soon be
  reworked to be a better experience
2023-07-12 16:27:39 +02:00
Daniel Lockyer
e893c92074 Deduplicated tsconfig.json files
refs https://github.com/TryGhost/DevOps/issues/45

- this moves the bulk of the tsconfig file to the `ghost/` folder and
  simply extends this file in each of the TS lib packages we currently
  have
- this makes it a lot easier to make single changes to our TS config
2023-07-12 09:34:00 +02:00
renovate[bot]
7dce046786 Update Test & linting packages 2023-07-11 15:26:07 +02:00
Naz
cdae2a978d Added ability to fetch collection posts by slug
closes https://github.com/TryGhost/Arch/issues/27

- We need a more convenient method of fetching posts belonging to a collection than by collection's "id". This change adds an alias to the existing endpoint `GET /collections/:id/posts/`. A non-valid ObjectID in the parameter is treated as a slug.
2023-07-11 09:30:01 +08:00
Naz
cb8e009c14 Allowed for index collection to have empty filter
refs https://github.com/TryGhost/Arch/issues/25

- The built in index (soon -> "latest") collection does not require any filtering so that it could contain an index of all posts in the system. All other automatic collections should have a filter defined.
2023-07-10 12:43:52 +08:00
renovate[bot]
aa8cbb9fa3 Update dependency typescript to v5.1.6 2023-06-29 08:24:30 +02:00
Fabien "egg" O'Carroll
16db3bbf17 Removed deletable property in favour of protected slugs
We couldn't decide on the best way to encode this in the database.
e.g. deletable: true? or type: internal? but type conflicts with the
storage of manual/automatic.

For now we can use this, and add something in future if we're finding problems
2023-06-29 01:40:05 +01:00
Fabien "egg" O'Carroll
cf83d169db Added use of a slug generator to offload calculation of slugs
When we end up wiring this to the database, this generator will also ensure
uniqueness by appending/incrementing a number on the end of the slug. Long term
it would be good to offload this to a shared slug service, this could also
ensure that slugs are unique globally or between multiple tables, if desired
2023-06-29 01:40:05 +01:00
Fabien "egg" O'Carroll
0a3e36cd62 Ensured uniqueness of slug in collections
We require that slugs are unique as slugs can/are used for routing purposes and
act as an identifier for a resource. As this is a core business rule, we want
to encode it in the entity so that it can be unit tested, and be enforced
regardless of underlying persistence layer
2023-06-29 01:40:05 +01:00
Fabien "egg" O'Carroll
d29f512823 Required titles for collections
We don't want to allow collections to be created without a title, and we need
to encoe that business rule in the entity.
2023-06-29 01:40:05 +01:00
Fabien "egg" O'Carroll
0cfa236570 Fixed visibility of internal _posts member
The _posts member is supposed to be an iternal property to store the data used
by the `posts` getter/setter
2023-06-29 01:40:05 +01:00