Summary:
Support custom dimensions
Had to move the definition of `Dimension` from `Duckling.Dimensions.Types` to `Duckling.Types` to avoid cyclic imports between these two modules.
A sample custom dimension is in `exe/CustomDimensionExample.hs`.
Limitations of custom dimensions:
- All rules for a custom dimension must be in the same module with the definition of the custom dimension. Otherwise there will be cyclic imports, because the definition of the dimension and the rules refer to each other.
- The custom dimension must be specified when using `parse`, since there's no way to get all the existing custom dimensions.
Reviewed By: patapizza
Differential Revision: D7630360
fbshipit-source-id: 30e12dcb33611f5692c4f5949de377bf61b75e1e
Summary:
Hello,
I have added new directory for AmountOfMoney NL Support
Added two files
- Duckling/AmountOfMoney/NL/Corpus.hs
- Duckling/AmountOfMoney/NL/Rules.hs
Updated File
- Duckling/Rules/NL.hs
Added test cases
- tests/Duckling/AmountOfMoney/NL/Tests.hs
Updated Test file
- tests/Duckling/AmountOfMoney/Tests.hs
Updated
duckling.cabal
Thanks for the review and the latest merge. Looking forward
Regards
Closes https://github.com/facebook/duckling/pull/173
Reviewed By: JonCoens
Differential Revision: D7592192
Pulled By: patapizza
fbshipit-source-id: 5895c29bf7f1033e4ffd791d5915a16d230e9375
Summary: Added ordinal support for Hindi
Reviewed By: patapizza
Differential Revision: D6944156
fbshipit-source-id: eb5da698e5cccde9a1cc31adf7bc433b89e07454
Summary:
Hi, in this pr:
* Added time dimension to Arabic language, thanks to Hussein-Dahir & Yazeed-Obaid for writing time corpus.
* Fixed some bugs in numeral & ordinals and added more test cases for them.
Also, I don't really understand why do we use classifiers in time dimension?
Closes https://github.com/facebook/duckling/pull/123
Reviewed By: blandinw
Differential Revision: D6583313
Pulled By: patapizza
fbshipit-source-id: f7acdef0c032d7b7fd7d224832fdaf484d2df825
Summary:
* `ruleIntegerNumeric` was used in all languages but Burmese.
* it seems like the hindu-arabic numerals are slowly getting in Burmese (e.g. recent car plates)
* Moving the rule in `Duckling/Numeral/Common.hs`
Reviewed By: blandinw
Differential Revision: D6498349
fbshipit-source-id: e868dc9960f18f0781e4aa98a0dfcd14969537c9
Summary:
This adds support for greek times.
There are still some issues with expressions of the form:
```
9:30 - 11:00 την πέμπτη
```
Where `11:00 την πέμπτη` is parsed first (as 11:30 on Thu), instead of prioritizing `9:30 - 11:00` as the training data suggests. These tests are for the moment excluded from the corpus.
Reviewed By: patapizza
Differential Revision: D6376271
fbshipit-source-id: 2f31e058fb88386429070e3b51cd33f93b9c5936
Summary:
- Added Duration dimension to Russian language
- Added TimeGrain dimension to Russian language
- Refactored isNatural and isNaturalWith out of Duration helpers into Numeral helpers
- Implemented <integer> and a half rule for Russian Numeral
- Changed the type of inSeconds to polymorphic one
Closes https://github.com/facebook/duckling/pull/105
Reviewed By: blandinw
Differential Revision: D6312604
Pulled By: patapizza
fbshipit-source-id: 9ae237b4beb6915ff8da013230457937d8e56733
Summary: I also did a check against nightly (which runs 8.2.1) and we were fine there too.
Reviewed By: blandinw
Differential Revision: D6169348
fbshipit-source-id: dd0727d
Summary: The date format changes between EN locales (https://en.wikipedia.org/wiki/Date_format_by_country). This diff fixes how dates are handled in each locale.
Reviewed By: patapizza
Differential Revision: D6156147
fbshipit-source-id: 22f296c
Summary:
* `en_CA` locale
* In Canada, Thanksgiving Day is the second Monday of October.
* Black Friday is the same as the US.
* However Canada observes both DDMM and MMDD formats. Defer to later, falling back to US.
Reviewed By: blandinw
Differential Revision: D6058909
fbshipit-source-id: 3d4e05e
Summary:
* Moving `ruleNationalDay` from `ZH` rules to specific locales: `zh_CN`, `zh_HK`, `zh_MO`
* Fixed National Day for `zh_TW`.
Reviewed By: blandinw
Differential Revision: D6057565
fbshipit-source-id: 8f9f2ab
Summary:
* Locales support for the library, following `<Lang>_<Region>` with ISO 639-1 code for `<Lang>` and ISO 3166-1 alpha-2 code for `<Region>` (#33)
* `Locale` opaque type (composite of `Lang` and `Region`) with `makeLocale` smart constructor to only allow valid `(Lang, Region)` combinations
* API: `Context`'s `lang` parameter has been replaced by `locale`, with optional `Region` and backward compatibility.
* `Rules/<Lang>.hs` exposes
- `langRules`: cross-locale rules for `<Lang>`, from `<Dimension>/<Lang>/Rules.hs`
- `localeRules`: locale-specific rules, from `<Dimension>/<Lang>/<Region>/Rules.hs`
- `defaultRules`: `langRules` + specific rules from select locales to ensure backward-compatibility
* Corpus, tests & classifiers
- 1 classifier per locale, with default classifier (`<Lang>_XX`) when no locale provided (backward-compatible)
- Default classifiers are built on existing corpus
- Locale classifiers are built on
- `<Dimension>/<Lang>/Corpus.hs` exposes a common `corpus` to all locales of `<Lang>`
- `<Dimension>/<Lang>/<Region>/Corpus.hs` exposes `allExamples`: a list of examples specific to the locale (following `<Dimension>/<Lang>/<Region>/Rules.hs`).
- Locale classifiers use the language corpus extended with the locale examples as training set.
- Locale examples need to use the same `Context` (i.e. reference time) as the language corpus.
- For backward compatibility, `<Dimension>/<Lang>/Corpus.hs` can expose also `defaultCorpus`, which is `corpus` augmented with specific examples. This is controlled by `getDefaultCorpusForLang` in `Duckling.Ranking.Generate`.
- Tests run against each classifier to make sure runtime works as expected.
* MM/DD (en_US) vs DD/MM (en_GB) example to illustrate
Reviewed By: JonCoens, blandinw
Differential Revision: D6038096
fbshipit-source-id: f29c28d
Summary:
- Fixes#89. "<number> <number>" would parse as Time if in the right range.
- Applied same rule for all languages. Note that for Italian and Polish, I updated "<hour> <minute>" tests to be in the form "at <hour> <minute>".
- Replaced `liftM2` with more generic `and|or . sequence [f1, f2, ...]`.
Reviewed By: blandinw
Differential Revision: D5992879
fbshipit-source-id: 5409ffb
Summary: Introducing Georgian (KA), and the very beginnings of numeral support
Reviewed By: patapizza
Differential Revision: D5757952
fbshipit-source-id: 89d05f8
Summary:
Repeating version bounds on executables that depend on
`duckling` library is needless bureaucracy.
Reviewed By: JonCoens
Differential Revision: D5478388
fbshipit-source-id: e18f1b8
Summary:
This adds two new targets:
* `:duckling-expensive` - meant to have inputs for which running Duckling is expensive
* `:duckling-request-sample` - meant to have a random sample of inputs
The reason to have 2 of them is that they measure different things.
`:duckling-expensive` is correlated with failures,
`:duckling-request-sample` is correlated with cost.
I intend to add basic instruction on how to use them for
benchmarking/profiling soon.
Reviewed By: patapizza
Differential Revision: D5301554
fbshipit-source-id: f73fd85
