ref https://linear.app/tryghost/issue/CFR-27
- updated packages to include performance improvement for NQL filter
strings including multiple neq filters for the same resource
- bumped `bookshelf-plugins`
- bumped NQL versions
We identified a performance fix that allows us to combine not equal
(neq) filters for the same resource in a logically-equivalent way that
also has far more performant resulting SQL.
We're effectively automatically combining strings like
'tag:-tag1+tag:-tag2` into 'tag:-[tag1,tag2]'.
ref https://linear.app/tryghost/issue/CFR-4/
- added request queueing middleware (express-queue) to handle high
request volume
- added new config option `optimization.requestQueue`
- added new config option `optimization.requestConcurrency`
- added logging of request queue depth - `req.queueDepth`
We've done a fair amount of investigation around improving Ghost's
resiliency to high request volume. While we believe this to be partly
due to database connection contention, it also seems Ghost gets
overwhelmed by the requests themselves. Implementing a simple queueing
system allows us a simple lever to change the volume of requests Ghost
is actually ingesting at any given time and gives us options besides
simply increasing database connection pool size.
---------
Co-authored-by: Michael Barrett <mike@ghost.org>
- this version is written in TS, but was published a few months ago and
needs to be bumped here
- also updates a previous deep include into the library, which was
unnecessary anyway
fixes GRO-25
Updated @tryghost/nql to 0.12.0 and other packages that depend on it
1. SQLite: when a filter string contains /.
When we use a NQL contain/starts/endsWith filter that contains a slash,
underlyingly the whole filter will get converted to a MongoDB query, in
which we just use a regexp to represent the filter. In here we will
escape the slash: \/ as expected in a regexp. Later when we convert this
MongoDB query back to knex/SQL, we use a SQL LIKE query. Currently we
don't remove the escaping here for a normal slash. MySQL seems to ignore
this (kinda incorrect). SQLite doesn't like it, and this breaks queries
on SQLite that use slashes. The solution here is simple: remove the
backslash escaping when converting the regexp to LIKE, just like we do
with other special regexp characters.
2. We don't escape % and _, which have a special meaning in LIKE queries
Usage of % and _ is now as expected and doesn't have the special SQL
meaning anymore.
refs https://github.com/TryGhost/Arch/issues/86
- When PostEditedEvent data contains no visible changes we can skip the matching collections update process alltogether. Each call to `updatePostInMatchingCollections` creates a transaction in addition to fetching all collections. There's no need to process anything when there are no relevant changes in the post edit!
refs https://github.com/TryGhost/Arch/issues/82
- Collections logs are too verbose causing noise.
- Moved some of the logs to use "debug" for now and made summarized logs for the information that we still need while collections code is actively monitored. The event info logs can be removed once we are passed the active phase of rolling out the collections feature
refs https://github.com/TryGhost/Arch/issues/46
- Similarly to post filters, collection filters now support both 'tag' and 'tags' nql filter keys when defining a filter for related tag slugs. For example, both `tag:avocado` and `tags:avocado` would both be valid collection filters that would filter by the same 'slug' property of the tags assigned to a post.
- Along with these changes had to rework the tags property of the collection posts to match the shape used in post resources. Moved from:
`tags: ['bacon', 'broc']`
to
`tags:[{slug: 'bacon'}, {slug: 'broc'}]`
refs https://github.com/TryGhost/Arch/issues/41
- When an new collection is created the relational "tags" filter is now picked up properly and appropriate posts matching the tag filter are assigned and stored in the collection. Example collection filter that is now supported: `tags:['bacon']`
- Additionally cleaned up returned collection post DTOs, so we return as little data as possible and add only the fields that are needed
refs https://github.com/TryGhost/DevOps/issues/45
- this switches the monorepo over to using Nx instead of Lerna, because
we don't currently need the versioning+publishing capabilities
- this also adds an `nx.json`, which allows us to enable task caching
- also adds `build:ts` to the TS projects, which is cached for fast execution
- how these interact with the dev.js script will hopefully soon be
reworked to be a better experience
We require that slugs are unique as slugs can/are used for routing purposes and
act as an identifier for a resource. As this is a core business rule, we want
to encode it in the entity so that it can be unit tested, and be enforced
regardless of underlying persistence layer
refs https://github.com/TryGhost/Team/issues/3169
- With the new architecture all possible dependencies should be injected rather than required directly - this applies to the DomainEvents module as well.
refs https://github.com/TryGhost/Team/issues/3169
- This piece of logic handles the 'post.added' model event mapping to Collection's PostAddedEvent domain event and logic related to updating collections when the new post is added.
refs https://github.com/TryGhost/Team/issues/3169
- To make the coupling to Ghost's model events as loose as possible added a bridge that maps model events to domain events. These domain events it what the collections module can subscribe to to make necessary updates.
Rather than using `require` we can add a type definition file, this allows us
to add custom types in the future if we want. We need to add some config for
ts-node to pick up the types correctly.
refs https://github.com/TryGhost/Team/issues/3294
- The factory method for the Collection and validations should live close together based on our latest architectural direction
refs https://github.com/TryGhost/Team/issues/3167
- The core client (API) needs a way to pass in information without an ID when creating a new entity, handling it on service/repository layer makes the most sense.
- Used "require" syntax to import tpl/errors modules, otherwise TS compiler was complaining about type compatibility issues, this works as a temporary workaround and is tracked and an issue to improve in the future.
closes https://github.com/TryGhost/Team/issues/3166
- The collections service contains CRUD logic to manage collection entities through: save, getById, getAll, and destroy methods.
