Summary:
Pull Request resolved: https://github.com/facebook/duckling/pull/533
In recent versions of Data.Some the name of the constructor, `This` has changed name to `Some`. This has become rather problematic for us to migrate so we're just going to remove the dependency. The meat of this diff is adding the type `Seal` to `Duckling.Types`. That type replaces `Some`.
Reviewed By: pepeiborra
Differential Revision: D23929459
fbshipit-source-id: 8ff4146ecba4f1119a17899961b2d877547f6e4f
Summary: Make test failure outputs readable by proper printing of `Data.Text`, using the `unpack` function rather than relying on the implementation of the `Show` typeclass for `Text`
Reviewed By: patapizza
Differential Revision: D18367058
fbshipit-source-id: b5aece3c8818f16dfe4c55235f6b9a183ba6f70f
Summary:
generalize chinese digit specifier (十百千万亿) parsing, and add hanzi tests
These digits specifiers can be parsed as (<num><speci>)<num>,
by using the multiplicater value <num><speci>, and a connect function that adds them together
(two cases, skipping digits [which requires a 零 in between], and digits in consecutive locations).
Note that 个 is technically a digit specifier,
but in Chinese it is never used directly as a numeral specifier, and always as a counter.
Reviewed By: zliu41
Differential Revision: D7424249
fbshipit-source-id: 20a85a7df1f908ee9879e92b904178fa26a9a5e5
Summary: show the input that an ambiguous parse is failing at
Reviewed By: patapizza
Differential Revision: D7502191
fbshipit-source-id: 9f0fbf8301413d9007236ba5b6af1f4b41c20269
Summary:
During ranking, due to how candidates are ordered, it is completely possible to have multiple correct candidates
have the exact same rank (equal range and exact equal score). In this case `analyze` returns all of them, which gets
misinterpreted as having multiple tokens in output rather than multiple solutions. Checks this case and gives the
correct prompt for ambiguous parses.
Reviewed By: patapizza
Differential Revision: D7489391
fbshipit-source-id: b66947e37bddb3ac6273843dd79b559aff9d0083
Summary: Add an option to return latent time entities. This can be used when one is pretty certain that the input contains a datetime.
Reviewed By: patapizza
Differential Revision: D7254245
fbshipit-source-id: e9e0503cace2691804056fcebdc18fd9090fb181
Summary:
* add flag for this/next/last time
* fix thanskgiving in EN locales
* `analyzedRangeTest` helper with `rangeTests` for `Time/EN`
Reviewed By: blandinw
Differential Revision: D6191209
fbshipit-source-id: 6eaa117
Summary:
* Locales support for the library, following `<Lang>_<Region>` with ISO 639-1 code for `<Lang>` and ISO 3166-1 alpha-2 code for `<Region>` (#33)
* `Locale` opaque type (composite of `Lang` and `Region`) with `makeLocale` smart constructor to only allow valid `(Lang, Region)` combinations
* API: `Context`'s `lang` parameter has been replaced by `locale`, with optional `Region` and backward compatibility.
* `Rules/<Lang>.hs` exposes
- `langRules`: cross-locale rules for `<Lang>`, from `<Dimension>/<Lang>/Rules.hs`
- `localeRules`: locale-specific rules, from `<Dimension>/<Lang>/<Region>/Rules.hs`
- `defaultRules`: `langRules` + specific rules from select locales to ensure backward-compatibility
* Corpus, tests & classifiers
- 1 classifier per locale, with default classifier (`<Lang>_XX`) when no locale provided (backward-compatible)
- Default classifiers are built on existing corpus
- Locale classifiers are built on
- `<Dimension>/<Lang>/Corpus.hs` exposes a common `corpus` to all locales of `<Lang>`
- `<Dimension>/<Lang>/<Region>/Corpus.hs` exposes `allExamples`: a list of examples specific to the locale (following `<Dimension>/<Lang>/<Region>/Rules.hs`).
- Locale classifiers use the language corpus extended with the locale examples as training set.
- Locale examples need to use the same `Context` (i.e. reference time) as the language corpus.
- For backward compatibility, `<Dimension>/<Lang>/Corpus.hs` can expose also `defaultCorpus`, which is `corpus` augmented with specific examples. This is controlled by `getDefaultCorpusForLang` in `Duckling.Ranking.Generate`.
- Tests run against each classifier to make sure runtime works as expected.
* MM/DD (en_US) vs DD/MM (en_GB) example to illustrate
Reviewed By: JonCoens, blandinw
Differential Revision: D6038096
fbshipit-source-id: f29c28d
Summary:
This works around https://github.com/haskell/cabal/issues/4350
If we don't do this files get compiled multiple times
and cabal is unhappy.
Reviewed By: patapizza
Differential Revision: D4782749
fbshipit-source-id: 5bbe425
