refs https://github.com/TryGhost/Toolbox/issues/430
- Previously the CSV parser had "map whatever you can and pass on unknown properties further" approach to CSV parsing. This logic has led to unwanted fields leaking through CSV imports - messy, dangerous.
- The strict mapping rules act as a "validator" to the user input, only passing through the fields we expect explicitly - safer clean cut solution with no unintended side-effects.
refs https://github.com/TryGhost/Toolbox/issues/430
refs https://github.com/TryGhost/Ghost/issues/14882
- Having an explicit mappings passed into the members CSV parser makes it easier to control and understand the transforms for package clients
- Eventually the parser will receive a strict map with the fields it should parse - skipping all unknown & unmapped fields
refs https://github.com/TryGhost/Toolbox/issues/430
- Not having any extra logic in the mapper will allow to have a generalized "mapping" concept for CSV input serialization
- This is groundwork for stricter header value filtering on the parsing stage
refs https://github.com/TryGhost/Toolbox/issues/430
- The 'readCSV' method was only exposed to be used in the unit tests. To keep the module code to the minimum moved readCSV to the unit test itself - the only place where it's used and belongs to.
fixes https://github.com/TryGhost/Team/issues/1911
Kept on creating a new error column when unparsing the error list. So for every error we would create a new column when it was already added.
no-issue
We have a special mapping for subscribed_to_emails -> subscribed in the
parse method, but were not mapping it in the unparse method, which meant
we were losing information during CSV imports.
refs https://github.com/TryGhost/Team/issues/1202
When importing we were transforming the CSV and add missing columns to
it before storing it in preparation to perform the import. This resulted
in the missing columns being updated for existing members with blank
data.
We've updated the Members CSV parsing library to take an options list of
columns to include, which then allows imports to not include all of the
default columns.
refs https://github.com/TryGhost/Team/issues/765
Support for multiple products means we can no longer map a members state
to a csv row using just the `complimentary_plan` option. Instead we must
include the product(s) that a member has. This ensures that we can read
and write this data from/to csv files.
no refs
The data structure for subscriptions object on member has changed in 4.x from` stripe.subscriptions` to direct `subscriptions`, the change here updates parsing of subscriptions data
no-issue
- Was not used by the importer and removed for simplicity.
- Updates the header mapping to happen in place, rather than in a loop
- Updates the parsing of values to give correct types
no issue
- When items are parsed from CSV empty values were interpreted as empty strings - ''. Empty strings are always transformed into 'null' values in Ghost's model layer and are much more problematic to validate comparing to plain `null`. Specifically validation was imossible for 'format: date-time' with JSON schema validation through ajv when the value of date property was an empty string
- This behavior resemples one present in Ghost's model layer - 95880dddeb
- When testing performance overhead for this change did not spot any statistically significant change in performance (tested set was 50K rows)
refs 5c46786ebc
- This is continuation of work removing csv-parser as main CSV handling library with more suitable papaparse library
- Referenced commit introduced papaparse as a library to serialize JSON to CSV, this changeset takes it a step further and replaces CSV to JSON seriazliation logic
no issue
- When processing CSV files `parse` function now allows for the client to specify "mapping" parameter in format of a hash as follows:
{ destination_property_name: 'source_column_name'}
e.g.:
{
name: 'weird_name_column',
email: 'email_column'
}
- It is done so to allow for the end user to provide exact mapping of the fields to be transformed into JSON.
no issue
- This fixes a problem where files are skiped form the @tryghost/members-csv package
- Also this follows the file structure convention with other packages
