Summary:
* Closes PRs #115, #116.
* `Numeral`: Moves `ruleFractions` to common
* `Numeral/PT`: fixes dot in regex
* `Time`: Moves `isNumeralSafeToUse` to `isIntegerBetween` (more robust and cleaner)
* `Time/EN`: refactored `ruleCycleLastNextN` to `ruleDurationLastNext` (so it also accounts for non-safe Numerals, like "in a few days")
Reviewed By: blandinw
Differential Revision: D6502958
fbshipit-source-id: 6d9c08a23dab88f441aadade08d663c8e0da415d
Summary:
* `ruleIntegerNumeric` was used in all languages but Burmese.
* it seems like the hindu-arabic numerals are slowly getting in Burmese (e.g. recent car plates)
* Moving the rule in `Duckling/Numeral/Common.hs`
Reviewed By: blandinw
Differential Revision: D6498349
fbshipit-source-id: e868dc9960f18f0781e4aa98a0dfcd14969537c9
Summary:
When I write "dois mil e duzentos" the result should be 2200, but duckling recognize the numbers separated and give the result:
`[{"dim":"number","body":"dois","value":{"value":2,"type":"value"},"start":0,"end":4},{"dim":"number","body":"mil","value":{"value":1000,"type":"value"},"start":5,"end":8},{"dim":"time","body":"mil","value":{"values":[],"value":"1000-01-01T00:00:00.000-07:53","grain":"year","type":"value"},"start":5,"end":8},{"dim":"number","body":"duzentos","value":{"value":200,"type":"value"},"start":11,"end":19}]`
Now with this commit, duckling gives the correct result:
`[{"dim":"number","body":"dois mil e duzentos","value":{"value":2200,"type":"value"},"start":0,"end":19}]`
Closes https://github.com/facebook/duckling/pull/117
Reviewed By: blandinw
Differential Revision: D6477925
Pulled By: patapizza
fbshipit-source-id: 26ab503cc8def739c51ceb5bae7546016ba65ad6
Summary:
This adds support for greek times.
There are still some issues with expressions of the form:
```
9:30 - 11:00 την πέμπτη
```
Where `11:00 την πέμπτη` is parsed first (as 11:30 on Thu), instead of prioritizing `9:30 - 11:00` as the training data suggests. These tests are for the moment excluded from the corpus.
Reviewed By: patapizza
Differential Revision: D6376271
fbshipit-source-id: 2f31e058fb88386429070e3b51cd33f93b9c5936
Summary:
When using speech recognition, we might see things like "six thirty six a m" or
"ten thirty p m".
Also fixed the argument order of `timeOfDayAMPM` to be more idiomatic.
Reviewed By: blandinw
Differential Revision: D6316542
fbshipit-source-id: 0008c049040219b3a1dd80d9e4661ba8a246fa7f
Summary:
"d" would parse as "dinsdag" (Tuesday), "zo" would parse as "zondag" (Sunday, also means "so").
Made dots mandatory, to prevent further issues (e.g. "zon" means "sun").
Reviewed By: mullender
Differential Revision: D6312693
fbshipit-source-id: 58c5824e3ff174fc9c293c3f2d13e152c60e51de
Summary:
* add flag for this/next/last time
* fix thanskgiving in EN locales
* `analyzedRangeTest` helper with `rangeTests` for `Time/EN`
Reviewed By: blandinw
Differential Revision: D6191209
fbshipit-source-id: 6eaa117
Summary: The date format changes between EN locales (https://en.wikipedia.org/wiki/Date_format_by_country). This diff fixes how dates are handled in each locale.
Reviewed By: patapizza
Differential Revision: D6156147
fbshipit-source-id: 22f296c
Summary:
One of the most common ways of saying "last \<day-of-the-week\>" in Swedish is saying "i \<day-of-the-week\>s", for example "i tisdags" (last Tuesday) or "i lördags" (last Saturday). This pull request adds support for this.
Closes https://github.com/facebookincubator/duckling/pull/92
Reviewed By: blandinw, kodafb
Differential Revision: D6064814
Pulled By: patapizza
fbshipit-source-id: 6ea5466
Summary:
`ter` also means `to have`, which is very common.
`ter` is a very rare form for `Tuesday`, only used in calendar-like contexts
Reviewed By: blandinw
Differential Revision: D6066506
fbshipit-source-id: c8cd231
Summary:
* `en_CA` locale
* In Canada, Thanksgiving Day is the second Monday of October.
* Black Friday is the same as the US.
* However Canada observes both DDMM and MMDD formats. Defer to later, falling back to US.
Reviewed By: blandinw
Differential Revision: D6058909
fbshipit-source-id: 3d4e05e
Summary:
* Moving `ruleNationalDay` from `ZH` rules to specific locales: `zh_CN`, `zh_HK`, `zh_MO`
* Fixed National Day for `zh_TW`.
Reviewed By: blandinw
Differential Revision: D6057565
fbshipit-source-id: 8f9f2ab
Summary:
* Locales support for the library, following `<Lang>_<Region>` with ISO 639-1 code for `<Lang>` and ISO 3166-1 alpha-2 code for `<Region>` (#33)
* `Locale` opaque type (composite of `Lang` and `Region`) with `makeLocale` smart constructor to only allow valid `(Lang, Region)` combinations
* API: `Context`'s `lang` parameter has been replaced by `locale`, with optional `Region` and backward compatibility.
* `Rules/<Lang>.hs` exposes
- `langRules`: cross-locale rules for `<Lang>`, from `<Dimension>/<Lang>/Rules.hs`
- `localeRules`: locale-specific rules, from `<Dimension>/<Lang>/<Region>/Rules.hs`
- `defaultRules`: `langRules` + specific rules from select locales to ensure backward-compatibility
* Corpus, tests & classifiers
- 1 classifier per locale, with default classifier (`<Lang>_XX`) when no locale provided (backward-compatible)
- Default classifiers are built on existing corpus
- Locale classifiers are built on
- `<Dimension>/<Lang>/Corpus.hs` exposes a common `corpus` to all locales of `<Lang>`
- `<Dimension>/<Lang>/<Region>/Corpus.hs` exposes `allExamples`: a list of examples specific to the locale (following `<Dimension>/<Lang>/<Region>/Rules.hs`).
- Locale classifiers use the language corpus extended with the locale examples as training set.
- Locale examples need to use the same `Context` (i.e. reference time) as the language corpus.
- For backward compatibility, `<Dimension>/<Lang>/Corpus.hs` can expose also `defaultCorpus`, which is `corpus` augmented with specific examples. This is controlled by `getDefaultCorpusForLang` in `Duckling.Ranking.Generate`.
- Tests run against each classifier to make sure runtime works as expected.
* MM/DD (en_US) vs DD/MM (en_GB) example to illustrate
Reviewed By: JonCoens, blandinw
Differential Revision: D6038096
fbshipit-source-id: f29c28d
Summary:
- Fixes#89. "<number> <number>" would parse as Time if in the right range.
- Applied same rule for all languages. Note that for Italian and Polish, I updated "<hour> <minute>" tests to be in the form "at <hour> <minute>".
- Replaced `liftM2` with more generic `and|or . sequence [f1, f2, ...]`.
Reviewed By: blandinw
Differential Revision: D5992879
fbshipit-source-id: 5409ffb
Summary: Consolidate HE language named-months in the same style of other language, particularly EN. Translations were taken from previous namedMonth# rules and merged into the combined rule. Hebrew-speaking verification would be helpful to verify the integrity of the translations, although they were already taken from the existing code.
Reviewed By: blandinw
Differential Revision: D5839625
fbshipit-source-id: d53f9c7
Summary: In Romanian, `sa` is fairly common: hai sa ne vedem (let's see), hai sa mergem (let's go).
Reviewed By: blandinw
Differential Revision: D5801345
fbshipit-source-id: db677e4
Summary: Introducing Georgian (KA), and the very beginnings of numeral support
Reviewed By: patapizza
Differential Revision: D5757952
fbshipit-source-id: 89d05f8
Summary:
Remove hardcoded named-days and named-months, and
replace them with ruleDaysOfWeek and ruleMonths.
Reviewed By: patapizza
Differential Revision: D5742209
fbshipit-source-id: 339fc0a
Summary:
The Galeic ruleset has 12 separate rules for months, and 7 for days. This
change replaces those with a list of months/days and a single function
to create a list of rules from those. This is the same approach as is currently in the English ruleset.
Reviewed By: patapizza
Differential Revision: D5756222
fbshipit-source-id: ac4bc42
Summary:
Combined each of seasons, instants, and holidays into a data list and a
function to generate the list of Rules.
*Instants = today, tomorrow, now, end of year, etc.
Reviewed By: patapizza
Differential Revision: D5730896
fbshipit-source-id: 23170e7
Summary:
The German ruleset has 12 separate rules for months, and 7 for days. This
change replaces those with a list of months/days and a single function
to create a list of rules from those. This is the same approach as is currently in the English ruleset.
Reviewed By: patapizza
Differential Revision: D5728656
fbshipit-source-id: 8590f4a
Summary: Consolidated all the days of week rules into one rule, and did the same for all the month rules.
Reviewed By: patapizza
Differential Revision: D5721202
fbshipit-source-id: 2b4a56f
Summary: Consolidated the previous days of weeks and month names in french duckling file to become only 2 rules. Allows for more concise, updated code.
Reviewed By: patapizza
Differential Revision: D5710056
fbshipit-source-id: 816ef88
Summary: Changed Danish time rules to use ruleDaysOfWeek and ruleMonths.
Reviewed By: patapizza
Differential Revision: D5709782
fbshipit-source-id: aa03065
Summary:
There are problems in the ordinal recognition for Swedish. The most severe one is that all the numbers above 15 are actually Danish, not Swedish. Apart from that digits and digits followed by a dot are considered ordinals.
This pull request fixes this and also adds support for ordinals up to 100. The structure of the code is similar to in the ordinal recognition in English. Tests are also updated, both the ordinal tests and the time tests where incorrect ordinals were used.
Closes https://github.com/facebookincubator/duckling/pull/86
Reviewed By: JonCoens
Differential Revision: D5698145
Pulled By: patapizza
fbshipit-source-id: c31d7bc
Summary: In Portuguese, "um" means the numeral "one" and the article "a".
Reviewed By: bfiss
Differential Revision: D5703396
fbshipit-source-id: 92ed04f
Summary:
Remove hardcoded named-days and named-months, and
replace them with ruleDaysOfWeek and ruleMonths.
Reviewed By: patapizza
Differential Revision: D5695475
fbshipit-source-id: d30557f
Summary: Consolidating the rules for months and days of the week in Italian following the pattern seen in English.
Reviewed By: patapizza
Differential Revision: D5665259
fbshipit-source-id: 45d6c3b
Summary:
We don't allow matches adjacent to a character of the same class.
We were treating uppercase and lowercase characters differently.
"jon Friday" wouldn't match "on" but "Jon Friday" would.
Reviewed By: blandinw
Differential Revision: D5653681
fbshipit-source-id: be67358
Summary: Added ruleIntervalDDDDMonth to EN to handle cases such as "23rd to 26th Oct" and "1-8 september"
Reviewed By: patapizza
Differential Revision: D5637280
fbshipit-source-id: a1fdcd2
Summary: Moved all named days to the same rule, moved all named months to the same rule. Kept same regexes, just consolidated them.
Reviewed By: patapizza
Differential Revision: D5637061
fbshipit-source-id: e08ecf9
Summary: Changed ruleIntervalMonthDDDD to use the ordinal predicate instead of ugly regex
Reviewed By: patapizza
Differential Revision: D5628188
fbshipit-source-id: 1dbe195
Summary: Added EN rule "ruleIntervalFromDDDDMonth" to support "from 10 to 16 August". Used "isDOMValue" helper rather than regex.
Reviewed By: patapizza
Differential Revision: D5610623
fbshipit-source-id: 00a5208
Summary: 'nie' means 'no' in Polish, and isn't a common abbreviation for 'niedziela' (Sunday).
Reviewed By: blandinw
Differential Revision: D5587036
fbshipit-source-id: bfda7fc
Summary:
Fixes#65.
* fixes US holidays
* Black Friday is actually the first day after Thanksgiving day (not necessary the fourth Friday of November)
Reviewed By: JonCoens
Differential Revision: D5533906
fbshipit-source-id: 1824cba
Summary:
In French, the form "at hh" is not valid (it requires an hour indicator).
This fixes false positives such as in "John a un rendez-vous."
Fixes https://github.com/wit-ai/wit/issues/666.
Reviewed By: JonCoens
Differential Revision: D5530713
fbshipit-source-id: ecee1e5
