closes https://github.com/TryGhost/Toolbox/issues/430
- The members importer used to import all fields present in the uploaded CSV if the headers match, even if they're not mapped in the UI. This behavior has lead to have misleading consequences and "hidden" features. For example, if the field was present but intentionally left as "Not imported" in the UI the field would still get imported.
- Having a strict list of supported import fields also allows for manageable long-term maintenance of the CSV Import API and detect/communicate changes when they happen.
- The list of the current default field mapping is:
email: 'email',
name: 'name',
note: 'note',
subscribed_to_emails: 'subscribed',
created_at: 'created_at',
complimentary_plan: 'complimentary_plan',
stripe_customer_id: 'stripe_customer_id',
labels: 'labels',
products: 'products'
refs https://github.com/TryGhost/Toolbox/issues/430
- To be able to introduce strict mapping rules (exclude unknown fields) we need to control the CSV header mapping on the importer level. This change moves the configuration up from CSV parser to the importer
- Also adds tests covering correct inserts for specially treated "subscribed_to_emails" field
refs https://github.com/TryGhost/Toolbox/issues/430
- "perform()" is what gets executed by the import job for both immediate import and "inline job" import. Testing it on granular level will allow to change it with more confidence when introducing strict field mapping rules
refs https://github.com/TryGhost/Toolbox/issues/430
- Importer code was filled with an unnecessarily complex "job" object that was passed around. It had an "id" property, which confusingly was a path to a file at all times.
- Simplified the logic significantly by keeping and passing around the path to a "prepared" members CSV.
refs https://github.com/TryGhost/Toolbox/issues/430
refs https://github.com/TryGhost/Ghost/issues/14882
- The MembersCSVImporter constructor is way to complex and needs refactoring. This complexity makes initialization in tests too bulky and makes tests hard to read.
- Having a builder method is a stopgap solution to avoid going into MembersCSVImporter refactoring too deep.
no issue
When importing members, the members-importer isn't aware of the context and therefore falls back to use `member` as a source in members event table. This makes it impossible to determine imported members from others.
- Added an `context` property to the options in the members-importer constructor
- Checked for `importer` context when creating member events and assigned the source `admin` to it
refs: https://github.com/TryGhost/Toolbox/issues/293
Things needed to create this:
* MemberSubscriptionEvent now has an import source
* Importer now creates events with this type
* Verification trigger logic changed to use 30 day window of imports
refs: https://github.com/TryGhost/Toolbox/issues/166
New package handles the email verification workflow to prevent spammers. It currently handles MembersSubscribeEvent to detect potential abuse of the API to add members, and exposes methods for checking the threshold / starting the verification process for use by other areas of the code (at the moment - just member imports).
The import package no longer needs to handle anything related to verification since it can be handled in the wrapper function in Ghost, and the API package doesn't need to do anything other than dispatch the new event.
refs https://github.com/TryGhost/Team/issues/1202
When importing we were transforming the CSV and add missing columns to
it before storing it in preparation to perform the import. This resulted
in the missing columns being updated for existing members with blank
data.
We've updated the Members CSV parsing library to take an options list of
columns to include, which then allows imports to not include all of the
default columns.
refs https://github.com/TryGhost/Team/issues/958
- The import threshold has to become a dynamic number based on external parameters. Because of this it makes most sense to have it as an async function parameter
- There's a package API change for importThreshold constructor paramter becoming a fetchThreshold async funciton parameter
refs https://github.com/TryGhost/Team/issues/912
- We need a way to check if the import threshold has been reached within the members importer. See more deets in refed issue
no issue
- After each test run there were csv files lefover which polluted the workspace.
- Ideally the write file would be some kind of a smart mock, so we'd won't have to do synchronous file remoal after each test but that's for later improvement if it becomes problematic
refs https://github.com/TryGhost/Team/issues/916
- These tests are nowhere close to perfect of even good. They barely pass "good enough" watermark... it's a start for new tests to be added easier in the future once the features are developed around CSV improrter
