refs https://github.com/TryGhost/Toolbox/issues/430
- Previously the CSV parser had "map whatever you can and pass on unknown properties further" approach to CSV parsing. This logic has led to unwanted fields leaking through CSV imports - messy, dangerous.
- The strict mapping rules act as a "validator" to the user input, only passing through the fields we expect explicitly - safer clean cut solution with no unintended side-effects.
refs https://github.com/TryGhost/Toolbox/issues/430
refs https://github.com/TryGhost/Ghost/issues/14882
- Having an explicit mappings passed into the members CSV parser makes it easier to control and understand the transforms for package clients
- Eventually the parser will receive a strict map with the fields it should parse - skipping all unknown & unmapped fields
refs https://github.com/TryGhost/Toolbox/issues/430
- Not having any extra logic in the mapper will allow to have a generalized "mapping" concept for CSV input serialization
- This is groundwork for stricter header value filtering on the parsing stage
refs https://github.com/TryGhost/Toolbox/issues/430
- The 'readCSV' method was only exposed to be used in the unit tests. To keep the module code to the minimum moved readCSV to the unit test itself - the only place where it's used and belongs to.
no-issue
- Was not used by the importer and removed for simplicity.
- Updates the header mapping to happen in place, rather than in a loop
- Updates the parsing of values to give correct types
no issue
- When items are parsed from CSV empty values were interpreted as empty strings - ''. Empty strings are always transformed into 'null' values in Ghost's model layer and are much more problematic to validate comparing to plain `null`. Specifically validation was imossible for 'format: date-time' with JSON schema validation through ajv when the value of date property was an empty string
- This behavior resemples one present in Ghost's model layer - 95880dddeb
- When testing performance overhead for this change did not spot any statistically significant change in performance (tested set was 50K rows)
refs 5c46786ebc
- This is continuation of work removing csv-parser as main CSV handling library with more suitable papaparse library
- Referenced commit introduced papaparse as a library to serialize JSON to CSV, this changeset takes it a step further and replaces CSV to JSON seriazliation logic
no issue
- When processing CSV files `parse` function now allows for the client to specify "mapping" parameter in format of a hash as follows:
{ destination_property_name: 'source_column_name'}
e.g.:
{
name: 'weird_name_column',
email: 'email_column'
}
- It is done so to allow for the end user to provide exact mapping of the fields to be transformed into JSON.
no issue
- This fixes a problem where files are skiped form the @tryghost/members-csv package
- Also this follows the file structure convention with other packages
refs 8185b42d9e5fd9e9051f08ce3395a648ec02f3a4
- Allows to move files and keep the history in one go.
- 'csv-parser' will be upgraded to 'papaparse' lib in a new package
