Summary:
Hi, in this pr:
* Added time dimension to Arabic language, thanks to Hussein-Dahir & Yazeed-Obaid for writing time corpus.
* Fixed some bugs in numeral & ordinals and added more test cases for them.
Also, I don't really understand why do we use classifiers in time dimension?
Closes https://github.com/facebook/duckling/pull/123
Reviewed By: blandinw
Differential Revision: D6583313
Pulled By: patapizza
fbshipit-source-id: f7acdef0c032d7b7fd7d224832fdaf484d2df825
Summary: Create hashmaps for the pattern matching rules, and make the regex matching read the value from the hashmap.
Reviewed By: patapizza
Differential Revision: D6592508
fbshipit-source-id: 0f4dd9bc548f71d0e6e52c938bd2e39bfd909c01
Summary: Create hashmaps for the pattern matching rules, and make the regex matching read the value from the hashmap.
Reviewed By: panagosg7
Differential Revision: D6536790
fbshipit-source-id: ecb0bf28cf22b82ef10c3c2408eda15409189431
Summary:
* Closes PRs #115, #116.
* `Numeral`: Moves `ruleFractions` to common
* `Numeral/PT`: fixes dot in regex
* `Time`: Moves `isNumeralSafeToUse` to `isIntegerBetween` (more robust and cleaner)
* `Time/EN`: refactored `ruleCycleLastNextN` to `ruleDurationLastNext` (so it also accounts for non-safe Numerals, like "in a few days")
Reviewed By: blandinw
Differential Revision: D6502958
fbshipit-source-id: 6d9c08a23dab88f441aadade08d663c8e0da415d
Summary:
* `ruleIntegerNumeric` was used in all languages but Burmese.
* it seems like the hindu-arabic numerals are slowly getting in Burmese (e.g. recent car plates)
* Moving the rule in `Duckling/Numeral/Common.hs`
Reviewed By: blandinw
Differential Revision: D6498349
fbshipit-source-id: e868dc9960f18f0781e4aa98a0dfcd14969537c9
Summary:
When I write "dois mil e duzentos" the result should be 2200, but duckling recognize the numbers separated and give the result:
`[{"dim":"number","body":"dois","value":{"value":2,"type":"value"},"start":0,"end":4},{"dim":"number","body":"mil","value":{"value":1000,"type":"value"},"start":5,"end":8},{"dim":"time","body":"mil","value":{"values":[],"value":"1000-01-01T00:00:00.000-07:53","grain":"year","type":"value"},"start":5,"end":8},{"dim":"number","body":"duzentos","value":{"value":200,"type":"value"},"start":11,"end":19}]`
Now with this commit, duckling gives the correct result:
`[{"dim":"number","body":"dois mil e duzentos","value":{"value":2200,"type":"value"},"start":0,"end":19}]`
Closes https://github.com/facebook/duckling/pull/117
Reviewed By: blandinw
Differential Revision: D6477925
Pulled By: patapizza
fbshipit-source-id: 26ab503cc8def739c51ceb5bae7546016ba65ad6
Summary:
This adds support for greek times.
There are still some issues with expressions of the form:
```
9:30 - 11:00 την πέμπτη
```
Where `11:00 την πέμπτη` is parsed first (as 11:30 on Thu), instead of prioritizing `9:30 - 11:00` as the training data suggests. These tests are for the moment excluded from the corpus.
Reviewed By: patapizza
Differential Revision: D6376271
fbshipit-source-id: 2f31e058fb88386429070e3b51cd33f93b9c5936
Summary: Duckling supports emails like "hey at things.com" but not "hey at things dot com". Let's add that.
Reviewed By: patapizza
Differential Revision: D6387471
fbshipit-source-id: 349a3a1ee55983daf5fda7579623f0c7bb32357f
Summary: Refactor days of week and months rules to avoid mindless copy-pasta.
Reviewed By: patapizza
Differential Revision: D6438622
fbshipit-source-id: 489261b53ba50f6624996ad4581c2bf1206a8cc1
Summary: Duckling supports emails like "hey at things.com" but not "hey at things dot com". Let's add that.
Reviewed By: patapizza
Differential Revision: D6364483
fbshipit-source-id: 3b7abaab879c44c30c80c928de72c202068894a8
Summary:
- Added Duration dimension to Russian language
- Added TimeGrain dimension to Russian language
- Refactored isNatural and isNaturalWith out of Duration helpers into Numeral helpers
- Implemented <integer> and a half rule for Russian Numeral
- Changed the type of inSeconds to polymorphic one
Closes https://github.com/facebook/duckling/pull/105
Reviewed By: blandinw
Differential Revision: D6312604
Pulled By: patapizza
fbshipit-source-id: 9ae237b4beb6915ff8da013230457937d8e56733
Summary:
When using speech recognition, we might see things like "six thirty six a m" or
"ten thirty p m".
Also fixed the argument order of `timeOfDayAMPM` to be more idiomatic.
Reviewed By: blandinw
Differential Revision: D6316542
fbshipit-source-id: 0008c049040219b3a1dd80d9e4661ba8a246fa7f
Summary:
Suggest to use wit.ai's built-in entities if running Duckling is not
an option.
Reviewed By: JonCoens
Differential Revision: D6312789
fbshipit-source-id: 08f1c5759cdd7761b456914cd5c42e1c111c6408
Summary:
"d" would parse as "dinsdag" (Tuesday), "zo" would parse as "zondag" (Sunday, also means "so").
Made dots mandatory, to prevent further issues (e.g. "zon" means "sun").
Reviewed By: mullender
Differential Revision: D6312693
fbshipit-source-id: 58c5824e3ff174fc9c293c3f2d13e152c60e51de
Summary: Some constructors for Time related rules are being reused among different language, so we can extract them in `Duckling/Time/Helpers.hs`.
Reviewed By: patapizza
Differential Revision: D6269106
fbshipit-source-id: 7d9969ce425ee27a2e1a32ea48932e16c7e6b1f1
Summary:
- Add more tests to Ar ordinal corpus
- Add HashMap lookups and add more rules to composite numbers
Closes https://github.com/facebook/duckling/pull/100
Reviewed By: JonCoens
Differential Revision: D6254264
Pulled By: patapizza
fbshipit-source-id: 6d50286f6bd3a60e34b0eb855fb38e044c3c39dc
Summary:
* add flag for this/next/last time
* fix thanskgiving in EN locales
* `analyzedRangeTest` helper with `rangeTests` for `Time/EN`
Reviewed By: blandinw
Differential Revision: D6191209
fbshipit-source-id: 6eaa117
Summary:
With the current Dockerfile, I get errors when trying to build:
```
Downloading lts-8.8 build plan ...
Downloaded lts-8.8 build plan.
Updating package index Hackage (mirrored at https://s3.amazonaws.com/hackage.fpcomplete.com/) ...
Selected mirror https://s3.amazonaws.com/hackage.fpcomplete.com/
Downloading root
Selected mirror https://s3.amazonaws.com/hackage.fpcomplete.com/
Downloading timestamp
Downloading snapshot
Downloading mirrors
Cannot update index (no local copy)
Downloading index
Updated package list downloaded
Populating index cache ...
Populated index cache.
Compiler version mismatched, found ghc-8.2.1 (x86_64), but expected minor version match with ghc-8.0.2 (x86_64) (based on resolver setting in /duckling/stack.yaml).
To install the correct GHC into /root/.stack/programs/x86_64-linux/, try running "stack setup" or use the "--install-ghc" flag.
The command '/bin/sh -c stack build' returned a non-zero code: 1
```
With these fixes, I run `stack setup` inside the docker image in case it was not setup previously on the host machine.
Closes https://github.com/facebookincubator/duckling/pull/98
Differential Revision: D6174261
Pulled By: patapizza
fbshipit-source-id: 4719d94
Summary: I also did a check against nightly (which runs 8.2.1) and we were fine there too.
Reviewed By: blandinw
Differential Revision: D6169348
fbshipit-source-id: dd0727d
Summary: The date format changes between EN locales (https://en.wikipedia.org/wiki/Date_format_by_country). This diff fixes how dates are handled in each locale.
Reviewed By: patapizza
Differential Revision: D6156147
fbshipit-source-id: 22f296c
