fixes https://github.com/TryGhost/Team/issues/2366
refs https://ghost.slack.com/archives/C02G9E68C/p1670232405014209
Probem described in issue.
In the old MEGA flow:
- The `email_verification_required` check is now repeated inside the job
In the new email service flow:
- The `email_verification_required` is now checked (didn't happen
before)
- When generating the email batch recipients, we only include members
that were created before the email was created. That way it is
impossible to avoid limit checks by inserting new members between
creating an email and sending an email.
- We don't need to repeat the check inside the job because of the above
changes
Improved handling of large imports:
- When checking `email_verification_required`, we now also check if the
import threshold is reached (a new method is introduced in
vertificationTrigger specifically for this usage). If it is, we start
the verification progress. This is required for long running imports
that only check the verification threshold at the very end.
- This change increases the concurrency of fastq to 3 (refs
https://ghost.slack.com/archives/C02G9E68C/p1670232405014209). So when
running a long import, it is now possible to send emails without having
to wait for the import. Above change makes sure it is not possible to
get around the verification limits.
Refactoring:
- Removed the need to use `updateVerificationTrigger` by making
thresholds getters instead of fixed variables.
- Improved awaiting of members import job in regression test
refs: https://github.com/TryGhost/Team/issues/1121
- This makes several key changes to the way errors are handled in the member importer, to ensure that we only show error messages to users that we wrote.
- Fundamentally, we no longer trust all API errors, and instead only trust a set of very specific API errors. Anything outside of that is replaced with a generic error message.
- Also switches the server-side error generated for email verification (which can throw during member import) to be a HostLimitError, as that is a more appropriate class.
- Note: there are many other parts of Ghost admin that need a similar overhaul, and a similar change we need to introduce server side to fully resolve the underlying issue of bubbling up code errors to the UI.
refs https://github.com/TryGhost/Team/issues/2192
The method signatures of the Event Repository have been updated to
take mongo filter objects, but this call-site was not updated.
Long term we should really be using NQL filter strings for our
filtering API and the mongo filter objects should be an implementation
detail, however we don't have time right now to rectify this.
closes https://github.com/TryGhost/Toolbox/issues/399
- The MemberCreatedEvent event is more accurate representation of the limit nature - counting the number of members created. The previous MemberSubscribeEvent was slightly hacky solution because a member could be subscribed/unsubscribed multiple times and distorting the limit counts.
refs https://github.com/TryGhost/Toolbox/issues/387
- The limit values should be as configurable as possible to adjust verification thresholds dinamically per-usecase. This solves a problem of doing a separate version release when we need to adjust the verification thresholds.
- Before this "importThreshold" was the same concept as "apiThreshold", which makes it hard&confusing to reason about and hard to parameterize each specific case.
refs https://github.com/TryGhost/Toolbox/issues/387
- Similar reasoning as to previous renames - the variables were named with a single trigger source in mind and now would be confusing with multiple verification trigger sources.
refs https://github.com/TryGhost/Toolbox/issues/387
- The "amountImported" was to specific to one verification trigger source. There can be multiple sources that start the verification process.
- Changed `startVerificationProcess` method signature to reflect it's a private method that's only used internally - exposed for testing purposes only.
refs https://github.com/TryGhost/Toolbox/issues/387
- There will three distinct verification limits soon. To keep the naming clear "configThreshold" would be too generic/confusing to use.
- Introduced jsdoc descriptions for the "source" parameter, which will be corelating with each new config parameter ("apiTriggerThreshold", "importTriggerThreshold", "adminTriggerThreshold", etc.). This should give a better visibility into parameters we are dealing in this area.
refs https://github.com/TryGhost/Toolbox/issues/387
- I'm about to add another event source - "admin". Before doing that made the method more parameter dependent, so it can handle limit triggering logic from multiple source and based on multiple configuration parameters.
refs https://github.com/TryGhost/Toolbox/issues/387
- The constructor should be light initialization logic only. Putting business logic into constructor is quite dirty and not really testable!
- if the threshold is Infinity, we shouldn't be loading the newsletter
subscription events because we are saying there is no threshold
- the code has a quick path to avoid comparing the values, but it still
loads the events upfront
- this commit moves the quick path up to return earlier
- this has the nice side-effect of producing 100% coverage on this
package
refs: https://github.com/TryGhost/Toolbox/issues/293
Things needed to create this:
* MemberSubscriptionEvent now has an import source
* Importer now creates events with this type
* Verification trigger logic changed to use 30 day window of imports
no-issue
There is a bug in the event repository code where filters are not
correctly applied, this results in the api verification being extremely
trigger happy. For now we're removing the logic, with the plan to revert
this commit once the event repository is fixed.
refs: https://github.com/TryGhost/Toolbox/issues/166
New package handles the email verification workflow to prevent spammers. It currently handles MembersSubscribeEvent to detect potential abuse of the API to add members, and exposes methods for checking the threshold / starting the verification process for use by other areas of the code (at the moment - just member imports).
The import package no longer needs to handle anything related to verification since it can be handled in the wrapper function in Ghost, and the API package doesn't need to do anything other than dispatch the new event.
