refs: https://github.com/TryGhost/Toolbox/issues/595
We're rolling out new rules around the node assert library, the first of which is enforcing the use of assert/strict. This means we don't need to use the strict version of methods, as the standard version will work that way by default.
This caught some gotchas in our existing usage of assert where the lack of strict mode had unexpected results:
- Url matching needs to be done on `url.href` see aa58b354a4
- Null and undefined are not the same thing, there were a few cases of this being confused
- Particularly questionable changes in [PostExporter tests](c1a468744b) tracked [here](https://github.com/TryGhost/Team/issues/3505).
- A typo see eaac9c293a
Moving forward, using assert strict should help us to catch unexpected behaviour, particularly around nulls and undefineds during implementation.
refs: https://github.com/TryGhost/Toolbox/issues/188
- some of our older packages used a pattern for linting which missed using test config for linting tests
- we need this to be consistent so that we can add more eslint rules for testing
- two packages also didn't use the lib pattern, which made the lint pattern error - so this was fixed as well
As discussed with the product team we want to enforce kebab-case file names for
all files, with the exception of files which export a single class, in which
case they should be PascalCase and reflect the class which they export.
This will help find classes faster, and should push better naming for them too.
Some files and packages have been excluded from this linting, specifically when
a library or framework depends on the naming of a file for the functionality
e.g. Ember, knex-migrator, adapter-manager
- we previously used `@stdlib/utils` instead of the child package
`@stdlib/copy`, which is a lot smaller and contains our only use of
the parent
- this saves 140+MB of dependencies
- we keep ending up with multiple versions of the depedency in our tree,
and it's causing problems when comparing instances
- the workaround I'm implementing for now is to bump the package
everywhere and set a resolution so we only have 1 shared instance
- hopefully we can come up with a better method down the line
no issue
This change makes the verification trigger behave closer to the original idea. As well as having a threshold to trigger email verification, the trigger should also rely on the number of "organic" / verified members. This means that large publications will not trigger the verification flow when they import members, but spammers / suspicious sites worth verifying will not have the organic member count to allow for larger imports.
- there's a weird situation when we have mixed versions of the
dependency because different libraries try to compare instances
- this brings the usage up to 1.2.21 so we can fix the build for now
fixes https://github.com/TryGhost/Team/issues/2366
refs https://ghost.slack.com/archives/C02G9E68C/p1670232405014209
Probem described in issue.
In the old MEGA flow:
- The `email_verification_required` check is now repeated inside the job
In the new email service flow:
- The `email_verification_required` is now checked (didn't happen
before)
- When generating the email batch recipients, we only include members
that were created before the email was created. That way it is
impossible to avoid limit checks by inserting new members between
creating an email and sending an email.
- We don't need to repeat the check inside the job because of the above
changes
Improved handling of large imports:
- When checking `email_verification_required`, we now also check if the
import threshold is reached (a new method is introduced in
vertificationTrigger specifically for this usage). If it is, we start
the verification progress. This is required for long running imports
that only check the verification threshold at the very end.
- This change increases the concurrency of fastq to 3 (refs
https://ghost.slack.com/archives/C02G9E68C/p1670232405014209). So when
running a long import, it is now possible to send emails without having
to wait for the import. Above change makes sure it is not possible to
get around the verification limits.
Refactoring:
- Removed the need to use `updateVerificationTrigger` by making
thresholds getters instead of fixed variables.
- Improved awaiting of members import job in regression test
- this was all getting terribly behind so I've done several things:
- majority of `@tryghost/*` except Lexical packages
- gscan + knex-migrator to remove old `@tryghost/errors` usage
- bumped lockfile
refs: https://github.com/TryGhost/Team/issues/1121
- This makes several key changes to the way errors are handled in the member importer, to ensure that we only show error messages to users that we wrote.
- Fundamentally, we no longer trust all API errors, and instead only trust a set of very specific API errors. Anything outside of that is replaced with a generic error message.
- Also switches the server-side error generated for email verification (which can throw during member import) to be a HostLimitError, as that is a more appropriate class.
- Note: there are many other parts of Ghost admin that need a similar overhaul, and a similar change we need to introduce server side to fully resolve the underlying issue of bubbling up code errors to the UI.
refs https://github.com/TryGhost/Team/issues/2192
The method signatures of the Event Repository have been updated to
take mongo filter objects, but this call-site was not updated.
Long term we should really be using NQL filter strings for our
filtering API and the mongo filter objects should be an implementation
detail, however we don't have time right now to rectify this.
closes https://github.com/TryGhost/Toolbox/issues/399
- The MemberCreatedEvent event is more accurate representation of the limit nature - counting the number of members created. The previous MemberSubscribeEvent was slightly hacky solution because a member could be subscribed/unsubscribed multiple times and distorting the limit counts.
refs https://github.com/TryGhost/Toolbox/issues/387
- The limit values should be as configurable as possible to adjust verification thresholds dinamically per-usecase. This solves a problem of doing a separate version release when we need to adjust the verification thresholds.
- Before this "importThreshold" was the same concept as "apiThreshold", which makes it hard&confusing to reason about and hard to parameterize each specific case.
refs https://github.com/TryGhost/Toolbox/issues/387
- Similar reasoning as to previous renames - the variables were named with a single trigger source in mind and now would be confusing with multiple verification trigger sources.
refs https://github.com/TryGhost/Toolbox/issues/387
- The "amountImported" was to specific to one verification trigger source. There can be multiple sources that start the verification process.
- Changed `startVerificationProcess` method signature to reflect it's a private method that's only used internally - exposed for testing purposes only.
refs https://github.com/TryGhost/Toolbox/issues/387
- There will three distinct verification limits soon. To keep the naming clear "configThreshold" would be too generic/confusing to use.
- Introduced jsdoc descriptions for the "source" parameter, which will be corelating with each new config parameter ("apiTriggerThreshold", "importTriggerThreshold", "adminTriggerThreshold", etc.). This should give a better visibility into parameters we are dealing in this area.
refs https://github.com/TryGhost/Toolbox/issues/387
- I'm about to add another event source - "admin". Before doing that made the method more parameter dependent, so it can handle limit triggering logic from multiple source and based on multiple configuration parameters.
refs https://github.com/TryGhost/Toolbox/issues/387
- The constructor should be light initialization logic only. Putting business logic into constructor is quite dirty and not really testable!
- cleaned up unused dependencies
- adds missing dependencies that are used in the code
- this should help us be more explicit about the dependencies a package
uses
- because of how the npm scripts were set up, we were running the full
Admin integration tests during the unit tests phase of CI
- this commit renames the majority of `test` to `test:unit` in the
package.json files, and aliases `test` to `test:unit`
- special packages like Admin have no-op'd `test:unit` scripts so we
don't end up running its tests
- if the threshold is Infinity, we shouldn't be loading the newsletter
subscription events because we are saying there is no threshold
- the code has a quick path to avoid comparing the values, but it still
loads the events upfront
- this commit moves the quick path up to return earlier
- this has the nice side-effect of producing 100% coverage on this
package
refs https://github.com/TryGhost/Toolbox/issues/354
- these READMEs were migrated over from when each package was in a
different repo
- they also assume you're going to be publishing the packages because it
mentions install instructions
- only a few of them contain custom content
- this commit deletes the majority of these files because they're now
not useful
- any that contained other instructions have been cut down
refs https://github.com/TryGhost/Toolbox/issues/354
- these repository links made sense when they were in different repos
and published to NPM but we don't publish these packages any more
- this commit deletes those keys from the files
- these were copied over during the monorepo conversion but we're not
going to be publishing these packages so the top-level LICENSE file
covers all packages here
- we're going to be pinning all dependencies within the monorepo
- this shouldn't change anything anyway because we're using the same
version across all packages
- these packages are split apart for local development, but will be
bundled into Ghost when publishing
- therefore, these packages won't be published so we are resetting the
versions to make them cleaner
