Summary:
This adds support for greek times.
There are still some issues with expressions of the form:
```
9:30 - 11:00 την πέμπτη
```
Where `11:00 την πέμπτη` is parsed first (as 11:30 on Thu), instead of prioritizing `9:30 - 11:00` as the training data suggests. These tests are for the moment excluded from the corpus.
Reviewed By: patapizza
Differential Revision: D6376271
fbshipit-source-id: 2f31e058fb88386429070e3b51cd33f93b9c5936
Summary: Duckling supports emails like "hey at things.com" but not "hey at things dot com". Let's add that.
Reviewed By: patapizza
Differential Revision: D6387471
fbshipit-source-id: 349a3a1ee55983daf5fda7579623f0c7bb32357f
Summary: Refactor days of week and months rules to avoid mindless copy-pasta.
Reviewed By: patapizza
Differential Revision: D6438622
fbshipit-source-id: 489261b53ba50f6624996ad4581c2bf1206a8cc1
Summary: Duckling supports emails like "hey at things.com" but not "hey at things dot com". Let's add that.
Reviewed By: patapizza
Differential Revision: D6364483
fbshipit-source-id: 3b7abaab879c44c30c80c928de72c202068894a8
Summary:
- Added Duration dimension to Russian language
- Added TimeGrain dimension to Russian language
- Refactored isNatural and isNaturalWith out of Duration helpers into Numeral helpers
- Implemented <integer> and a half rule for Russian Numeral
- Changed the type of inSeconds to polymorphic one
Closes https://github.com/facebook/duckling/pull/105
Reviewed By: blandinw
Differential Revision: D6312604
Pulled By: patapizza
fbshipit-source-id: 9ae237b4beb6915ff8da013230457937d8e56733
Summary:
When using speech recognition, we might see things like "six thirty six a m" or
"ten thirty p m".
Also fixed the argument order of `timeOfDayAMPM` to be more idiomatic.
Reviewed By: blandinw
Differential Revision: D6316542
fbshipit-source-id: 0008c049040219b3a1dd80d9e4661ba8a246fa7f
Summary:
Suggest to use wit.ai's built-in entities if running Duckling is not
an option.
Reviewed By: JonCoens
Differential Revision: D6312789
fbshipit-source-id: 08f1c5759cdd7761b456914cd5c42e1c111c6408
Summary:
"d" would parse as "dinsdag" (Tuesday), "zo" would parse as "zondag" (Sunday, also means "so").
Made dots mandatory, to prevent further issues (e.g. "zon" means "sun").
Reviewed By: mullender
Differential Revision: D6312693
fbshipit-source-id: 58c5824e3ff174fc9c293c3f2d13e152c60e51de
Summary: Some constructors for Time related rules are being reused among different language, so we can extract them in `Duckling/Time/Helpers.hs`.
Reviewed By: patapizza
Differential Revision: D6269106
fbshipit-source-id: 7d9969ce425ee27a2e1a32ea48932e16c7e6b1f1
Summary:
- Add more tests to Ar ordinal corpus
- Add HashMap lookups and add more rules to composite numbers
Closes https://github.com/facebook/duckling/pull/100
Reviewed By: JonCoens
Differential Revision: D6254264
Pulled By: patapizza
fbshipit-source-id: 6d50286f6bd3a60e34b0eb855fb38e044c3c39dc
Summary:
* add flag for this/next/last time
* fix thanskgiving in EN locales
* `analyzedRangeTest` helper with `rangeTests` for `Time/EN`
Reviewed By: blandinw
Differential Revision: D6191209
fbshipit-source-id: 6eaa117
Summary:
With the current Dockerfile, I get errors when trying to build:
```
Downloading lts-8.8 build plan ...
Downloaded lts-8.8 build plan.
Updating package index Hackage (mirrored at https://s3.amazonaws.com/hackage.fpcomplete.com/) ...
Selected mirror https://s3.amazonaws.com/hackage.fpcomplete.com/
Downloading root
Selected mirror https://s3.amazonaws.com/hackage.fpcomplete.com/
Downloading timestamp
Downloading snapshot
Downloading mirrors
Cannot update index (no local copy)
Downloading index
Updated package list downloaded
Populating index cache ...
Populated index cache.
Compiler version mismatched, found ghc-8.2.1 (x86_64), but expected minor version match with ghc-8.0.2 (x86_64) (based on resolver setting in /duckling/stack.yaml).
To install the correct GHC into /root/.stack/programs/x86_64-linux/, try running "stack setup" or use the "--install-ghc" flag.
The command '/bin/sh -c stack build' returned a non-zero code: 1
```
With these fixes, I run `stack setup` inside the docker image in case it was not setup previously on the host machine.
Closes https://github.com/facebookincubator/duckling/pull/98
Differential Revision: D6174261
Pulled By: patapizza
fbshipit-source-id: 4719d94
Summary: I also did a check against nightly (which runs 8.2.1) and we were fine there too.
Reviewed By: blandinw
Differential Revision: D6169348
fbshipit-source-id: dd0727d
Summary: The date format changes between EN locales (https://en.wikipedia.org/wiki/Date_format_by_country). This diff fixes how dates are handled in each locale.
Reviewed By: patapizza
Differential Revision: D6156147
fbshipit-source-id: 22f296c
Summary:
One of the most common ways of saying "last \<day-of-the-week\>" in Swedish is saying "i \<day-of-the-week\>s", for example "i tisdags" (last Tuesday) or "i lördags" (last Saturday). This pull request adds support for this.
Closes https://github.com/facebookincubator/duckling/pull/92
Reviewed By: blandinw, kodafb
Differential Revision: D6064814
Pulled By: patapizza
fbshipit-source-id: 6ea5466
Summary:
`ter` also means `to have`, which is very common.
`ter` is a very rare form for `Tuesday`, only used in calendar-like contexts
Reviewed By: blandinw
Differential Revision: D6066506
fbshipit-source-id: c8cd231
Summary:
* `en_CA` locale
* In Canada, Thanksgiving Day is the second Monday of October.
* Black Friday is the same as the US.
* However Canada observes both DDMM and MMDD formats. Defer to later, falling back to US.
Reviewed By: blandinw
Differential Revision: D6058909
fbshipit-source-id: 3d4e05e
Summary:
* Moving `ruleNationalDay` from `ZH` rules to specific locales: `zh_CN`, `zh_HK`, `zh_MO`
* Fixed National Day for `zh_TW`.
Reviewed By: blandinw
Differential Revision: D6057565
fbshipit-source-id: 8f9f2ab
Summary:
* Locales support for the library, following `<Lang>_<Region>` with ISO 639-1 code for `<Lang>` and ISO 3166-1 alpha-2 code for `<Region>` (#33)
* `Locale` opaque type (composite of `Lang` and `Region`) with `makeLocale` smart constructor to only allow valid `(Lang, Region)` combinations
* API: `Context`'s `lang` parameter has been replaced by `locale`, with optional `Region` and backward compatibility.
* `Rules/<Lang>.hs` exposes
- `langRules`: cross-locale rules for `<Lang>`, from `<Dimension>/<Lang>/Rules.hs`
- `localeRules`: locale-specific rules, from `<Dimension>/<Lang>/<Region>/Rules.hs`
- `defaultRules`: `langRules` + specific rules from select locales to ensure backward-compatibility
* Corpus, tests & classifiers
- 1 classifier per locale, with default classifier (`<Lang>_XX`) when no locale provided (backward-compatible)
- Default classifiers are built on existing corpus
- Locale classifiers are built on
- `<Dimension>/<Lang>/Corpus.hs` exposes a common `corpus` to all locales of `<Lang>`
- `<Dimension>/<Lang>/<Region>/Corpus.hs` exposes `allExamples`: a list of examples specific to the locale (following `<Dimension>/<Lang>/<Region>/Rules.hs`).
- Locale classifiers use the language corpus extended with the locale examples as training set.
- Locale examples need to use the same `Context` (i.e. reference time) as the language corpus.
- For backward compatibility, `<Dimension>/<Lang>/Corpus.hs` can expose also `defaultCorpus`, which is `corpus` augmented with specific examples. This is controlled by `getDefaultCorpusForLang` in `Duckling.Ranking.Generate`.
- Tests run against each classifier to make sure runtime works as expected.
* MM/DD (en_US) vs DD/MM (en_GB) example to illustrate
Reviewed By: JonCoens, blandinw
Differential Revision: D6038096
fbshipit-source-id: f29c28d
Summary:
- Fixes#89. "<number> <number>" would parse as Time if in the right range.
- Applied same rule for all languages. Note that for Italian and Polish, I updated "<hour> <minute>" tests to be in the form "at <hour> <minute>".
- Replaced `liftM2` with more generic `and|or . sequence [f1, f2, ...]`.
Reviewed By: blandinw
Differential Revision: D5992879
fbshipit-source-id: 5409ffb
Summary: Replaced pattern matching with Hashmap. Also, removed ruleInteger17 and moved its regex to ruleInteger.
Reviewed By: patapizza
Differential Revision: D5812629
fbshipit-source-id: f0c1a06
