Commit Graph

86 Commits

Author SHA1 Message Date
Gerben Janssen van Doorn
f70f991b38 Added support Lao numerals
Summary: Duckling didn't support Lao numerals, this diff adds it.

Reviewed By: patapizza

Differential Revision: D9323242

fbshipit-source-id: 5dad0d4dfb4843281a327947690e664c91ab8f1a
2018-08-17 10:31:17 -07:00
PhalPheaktra Chhaya
b541354c31 Add Numeral dimension for new language KM. (#234)
Summary:
Hello,
I am new to Haskell, but I would like to add Khmer language (KM) to Duckling.
I have tried to extended Duckling by adding Numeral dimension for new language KM.
Please have a look at it and see what we can improve.

Thanks!
Pull Request resolved: https://github.com/facebook/duckling/pull/234

Reviewed By: blandinw

Differential Revision: D9032639

Pulled By: chinmay87

fbshipit-source-id: 7db19edf732fe6500629cc89e18e0655d7bbc48b
2018-08-03 14:45:55 -07:00
Nivin Lawrence
e279c5642e added tamil ordinal
Summary: support tamil ordinal

Reviewed By: girifb

Differential Revision: D8713963

fbshipit-source-id: c0760f0cf13d983473d29e18425ba926584c3072
2018-07-16 17:15:53 -07:00
Arunavha Chanda
d5555d0149 Numeral/BN: Adding Bengali numeral support to Duckling
Summary: Added support for Bengali numerals

Reviewed By: patapizza

Differential Revision: D8730468

fbshipit-source-id: dc36017e24d796f35abc477a0b8b317218c64a6a
2018-07-09 12:30:30 -07:00
Tero Laxström
4ed1ed83ed Basics for Finnish (#210)
Summary:
Adds Locale and Numeral for Finnish
Closes https://github.com/facebook/duckling/pull/210

Reviewed By: JonCoens

Differential Revision: D8430386

Pulled By: patapizza

fbshipit-source-id: a3c8b3b3419b7f43e2ef332cdb1fb8fc07da3bec
2018-06-19 10:45:27 -07:00
Anshuman Chhabra
5ab49060e1 Added Temperature support for Hindi (HI) language (#206)
Summary:
This PR adds Temperature dimension support for Hindi (HI). Let me know if there are any changes to be made :)

Cheers!
Closes https://github.com/facebook/duckling/pull/206

Reviewed By: haoxuany

Differential Revision: D8381472

Pulled By: patapizza

fbshipit-source-id: a085ba73910a80a51385f01475d9ba75f72fd4c5
2018-06-13 17:00:35 -07:00
Julien Odent
f272a175d4 Time/NL: Fix Sinterklaas for Belgium
Summary: In south Netherlands and in Belgium, Sinterklaas is celebrated on December 6.

Reviewed By: mullender

Differential Revision: D8281767

fbshipit-source-id: 4fc71783b9f543c4dfb04b8ba8481935cfb30dd8
2018-06-05 16:00:43 -07:00
Anshuman Chhabra
3e085fcfb1 Added TimeGrain to Hindi (HI)
Summary:
I have added __TimeGrain__ support to __Hindi (HI)__. Please let me know if there are any other changes required!

Cheers! :)
Closes https://github.com/facebook/duckling/pull/198

Reviewed By: patapizza

Differential Revision: D8082131

Pulled By: chinmay87

fbshipit-source-id: 73b8263a9f95775c32905edede14113c51b10431
2018-05-22 12:15:32 -07:00
RIAN DOUGLAS
884904b5ca Implement handling of "grand" for EN_?? locales
Summary: Add implementation of "a grand" and "<num> grand" for AU, BZ, CA, GB, IE, IN, JM, NZ, PH, TT, ZA locales. Some resolve to the local currency (AU, IN), others resolve to Dollar (NZ, PH).

Reviewed By: patapizza

Differential Revision: D7943186

fbshipit-source-id: c71ab462fa9df0ee65223ee82dc2c98457a4e13b
2018-05-17 15:45:27 -07:00
Anshuman Chhabra
8237ef4503 Added Duration support to Hindi (HI)
Summary:
Hello!

I have added __Hindi (HI)__ support for the __Duration__ dimension in ```Duckling/Duration/Rules.hs```. Written tests as well in ```Duckling/Duration/Corpus.hs``` and everything is passing and verified (`253 tests passed`).

Cheers!
Closes https://github.com/facebook/duckling/pull/194

Reviewed By: patapizza

Differential Revision: D7999571

Pulled By: chinmay87

fbshipit-source-id: deb5d60ba7f7ecbc2aa4d97ce5fb96e9bbe63b3d
2018-05-17 14:46:21 -07:00
RIAN DOUGLAS
963aa6b465 Implement handling of "grand" for Ca and GB
Summary: Add implementation of "a grand" and "<num> grand" for EN_CA and EN_GB locales

Reviewed By: patapizza

Differential Revision: D7916643

fbshipit-source-id: 0cd55f17ec522c0334f48436a8a8cc19e0560b0b
2018-05-09 14:46:16 -07:00
Ziyang Liu
c84166d708 Add Quantity/ZH
Summary:
Fixes #181.
Closes https://github.com/facebook/duckling/pull/182

Reviewed By: patapizza

Differential Revision: D7891288

Pulled By: haoxuany

fbshipit-source-id: faa505dbf149c728545a92579cb2e92a89cd5cb8
2018-05-07 09:45:32 -07:00
RIAN DOUGLAS
f328821bbc Add support for using 'grand' to refer to 1,000's of currency in EN
Summary: Added support for ",number> grand" and "a grand" to result in thousands of currency amount

Reviewed By: haoxuany

Differential Revision: D7688809

fbshipit-source-id: 72a81c7a1c48329f85c9f525b72b00c479a9edb0
2018-05-03 14:45:25 -07:00
Ziyang Liu
5460d8df0e Support custom dimensions
Summary:
Support custom dimensions

Had to move the definition of `Dimension` from `Duckling.Dimensions.Types` to `Duckling.Types` to avoid cyclic imports between these two modules.

A sample custom dimension is in `exe/CustomDimensionExample.hs`.

Limitations of custom dimensions:

- All rules for a custom dimension must be in the same module with the definition of the custom dimension. Otherwise there will be cyclic imports, because the definition of the dimension and the rules refer to each other.
- The custom dimension must be specified when using `parse`, since there's no way to get all the existing custom dimensions.

Reviewed By: patapizza

Differential Revision: D7630360

fbshipit-source-id: 30e12dcb33611f5692c4f5949de377bf61b75e1e
2018-04-19 15:30:51 -07:00
Aaron Yue
4df76289bc Implement AmountOfMoney
Summary: implement AmountOfMoney rules and corpus for ZH

Reviewed By: patapizza

Differential Revision: D7508507

fbshipit-source-id: 3591b399a9880c5278587979c6576720343cc123
2018-04-17 16:45:32 -07:00
Souvik Ghosh
ced73dcdcb AmountOfMoney NL Support
Summary:
Hello,
I have added new directory for AmountOfMoney NL Support
Added two files
- Duckling/AmountOfMoney/NL/Corpus.hs
- Duckling/AmountOfMoney/NL/Rules.hs

Updated File
- Duckling/Rules/NL.hs

Added test cases
- tests/Duckling/AmountOfMoney/NL/Tests.hs

Updated Test file
- tests/Duckling/AmountOfMoney/Tests.hs

Updated
duckling.cabal

Thanks for the review and the latest merge. Looking forward

Regards
Closes https://github.com/facebook/duckling/pull/173

Reviewed By: JonCoens

Differential Revision: D7592192

Pulled By: patapizza

fbshipit-source-id: 5895c29bf7f1033e4ffd791d5915a16d230e9375
2018-04-11 17:15:30 -07:00
Ziyang Liu
22d11c055b Add ZH distance support
Summary: Add ZH distance support

Reviewed By: patapizza

Differential Revision: D7341977

fbshipit-source-id: 301d90515c492da9fa4c4faf5bcc3351eacbed1b
2018-03-21 12:15:28 -07:00
hend
f70b4ebe07 add distance to SV
Summary: Closes https://github.com/facebook/duckling/pull/168

Reviewed By: chinmay87

Differential Revision: D7322559

Pulled By: patapizza

fbshipit-source-id: 3d4b86068c05a7a82442bc7c182b80b5cf75ec90
2018-03-21 11:15:25 -07:00
yasen-yankov
21c3b32e4d Added TimeGrain, Duration, and Ordinal
Summary: Closes https://github.com/facebook/duckling/pull/164

Reviewed By: chinmay87

Differential Revision: D7280110

Pulled By: patapizza

fbshipit-source-id: d98ddd900fe83f06b28afd39ea3311f42716288c
2018-03-19 18:00:29 -07:00
Giri Anantharaman
519c9519a3 Support Tamil numerals
Summary:
* Setup Tamil (TA) language
* Added Numeral Dimension

Reviewed By: patapizza

Differential Revision: D7323636

fbshipit-source-id: 4b1a42197ff4799880cded9ce86b8d7fae1507bc
2018-03-19 16:45:36 -07:00
Julien Odent
490799fe4a Time/EN: Periodic holidays
Summary:
* added periodic holidays for all existing English locales
* moved Independence Day from common to US, as it is country-dependent
* moved Memorial Day from common to US, as it is country-dependent (fixed the one for Canada)
* added Decoration Day as an alias for US Memorial Day

Reviewed By: chinmay87

Differential Revision: D7294129

fbshipit-source-id: 89cdbf91b4c43e1996fa1f4509eac1d1b1978197
2018-03-16 17:17:08 -07:00
yasen-yankov
1493d44465 Added BG distance
Summary: Closes https://github.com/facebook/duckling/pull/162

Reviewed By: chinmay87

Differential Revision: D7239243

Pulled By: patapizza

fbshipit-source-id: a5518219a67fa46bb06e97eee3dfd07ab683162f
2018-03-13 15:45:32 -07:00
bidhan-a
43079e7113 Setup Nepali (NE) and add Numeral dimension
Summary:
- Setup Nepali (NE) language
- Add basic Numeral dimension
Closes https://github.com/facebook/duckling/pull/156

Reviewed By: JonCoens

Differential Revision: D6965558

Pulled By: patapizza

fbshipit-source-id: f46c9b104d4345f20bd0cf53f8c9c8754855f314
2018-02-13 07:45:31 -08:00
Ashwini Reddy Challa
c8501d3e85 Added HI Ordinals
Summary: Added ordinal support for Hindi

Reviewed By: patapizza

Differential Revision: D6944156

fbshipit-source-id: eb5da698e5cccde9a1cc31adf7bc433b89e07454
2018-02-09 13:15:30 -08:00
Julien Odent
bef7a44fa8 Remove redundant brackets and language pragmas
Summary: .

Reviewed By: JonCoens

Differential Revision: D6838082

fbshipit-source-id: 94757bdb80c6d3c29a7a6554429940a1b7403108
2018-01-29 16:45:28 -08:00
Julien Odent
7f21d1d7d7 Time/EN: Parse MM.DD.YYYY (or DD.MM.YYYY)
Summary:
Fixes #143.

Doesn't allow for MM.DD.YY (or DD.MM.YY) as this clashes with HHMMSS.

Reviewed By: blandinw

Differential Revision: D6765534

fbshipit-source-id: 1d1720e5bb292fdb3970e76475f35eab316048a8
2018-01-19 19:15:49 -08:00
Abdallatif Sulaiman
89822776c6 Added Amount of Money Dimension to Arabic language
Summary: Closes https://github.com/facebook/duckling/pull/134

Reviewed By: JonCoens

Differential Revision: D6649920

Pulled By: patapizza

fbshipit-source-id: 9c647c84f5ae4f3dc26cb0c7aa74abb097ea001a
2018-01-02 14:00:51 -08:00
Abdallatif Sulaiman
2d726d3837 Added Quantity Dimension to Arabic language
Summary: Closes https://github.com/facebook/duckling/pull/127

Reviewed By: panagosg7

Differential Revision: D6616298

Pulled By: patapizza

fbshipit-source-id: dc774ff1e870bbd083a9cca8ee6f75db852afce9
2017-12-20 18:00:43 -08:00
Abdallatif Sulaiman
c056a0b46a Added Temperature Dimension to Arabic language
Summary: Closes https://github.com/facebook/duckling/pull/126

Reviewed By: panagosg7

Differential Revision: D6614995

Pulled By: patapizza

fbshipit-source-id: 35ff142a3fc8a498d9abbb80d28dc9be6cdcbc4d
2017-12-20 18:00:43 -08:00
Abdallatif Sulaiman
380457db8f Added Volume Dimension to Arabic language
Summary: Closes https://github.com/facebook/duckling/pull/125

Reviewed By: panagosg7

Differential Revision: D6614574

Pulled By: patapizza

fbshipit-source-id: 054ed04e0cc1cf79be31340ebb8b7fea3bc67f57
2017-12-20 18:00:43 -08:00
Abdallatif Sulaiman
1393098bcc Added Time Dimension to Arabic
Summary:
Hi, in this pr:
* Added time dimension to Arabic language, thanks to Hussein-Dahir & Yazeed-Obaid for writing time corpus.
* Fixed some bugs in numeral & ordinals and added more test cases for them.

Also, I don't really understand why do we use classifiers in time dimension?
Closes https://github.com/facebook/duckling/pull/123

Reviewed By: blandinw

Differential Revision: D6583313

Pulled By: patapizza

fbshipit-source-id: f7acdef0c032d7b7fd7d224832fdaf484d2df825
2017-12-19 14:30:42 -08:00
Newinfinite007
c133bad24a Hindi Language Numeral Dimension(minimalistic model). Tests passed.
Summary: Closes https://github.com/facebook/duckling/pull/119

Reviewed By: JonCoens

Differential Revision: D6597628

Pulled By: patapizza

fbshipit-source-id: 8bac0f686d6cecc38d9998e37042fe48f73530dc
2017-12-19 13:15:30 -08:00
Zahar Shimanchik
2274e40369 Added AmountOfMoney Dimension to Russian language
Summary: Closes https://github.com/facebook/duckling/pull/120

Reviewed By: patapizza

Differential Revision: D6520302

Pulled By: panagosg7

fbshipit-source-id: 2c9ac6e15ca3ee90b2bf5911ee62835966fcacd1
2017-12-12 12:15:30 -08:00
Julien Odent
6df3b26707 Numeral: common rule + supporting hindu-arabic numerals for Burmese
Summary:
* `ruleIntegerNumeric` was used in all languages but Burmese.
* it seems like the hindu-arabic numerals are slowly getting in Burmese (e.g. recent car plates)
* Moving the rule in `Duckling/Numeral/Common.hs`

Reviewed By: blandinw

Differential Revision: D6498349

fbshipit-source-id: e868dc9960f18f0781e4aa98a0dfcd14969537c9
2017-12-06 16:00:28 -08:00
Panagiotis Vekris
12a726aee7 Support for Greek times and dates
Summary:
This adds support for greek times.

There are still some issues with expressions of the form:
```
9:30 - 11:00 την πέμπτη
```
Where `11:00 την πέμπτη` is parsed first (as 11:30 on Thu), instead of prioritizing `9:30 - 11:00` as the training data suggests. These tests are for the moment excluded from the corpus.

Reviewed By: patapizza

Differential Revision: D6376271

fbshipit-source-id: 2f31e058fb88386429070e3b51cd33f93b9c5936
2017-12-04 16:45:40 -08:00
igor-drozdov
29d776dee5 Added TimeGrain and Duration Dimensions to Russian language
Summary:
- Added Duration dimension to Russian language
- Added TimeGrain dimension to Russian language
- Refactored isNatural and isNaturalWith out of Duration helpers into Numeral helpers
- Implemented <integer> and a half rule for Russian Numeral
- Changed the type of inSeconds to polymorphic one
Closes https://github.com/facebook/duckling/pull/105

Reviewed By: blandinw

Differential Revision: D6312604

Pulled By: patapizza

fbshipit-source-id: 9ae237b4beb6915ff8da013230457937d8e56733
2017-11-15 10:45:24 -08:00
Igor Drozdov
f6492b5da0 Added Quantity dimension to Russian language
Summary: Closes https://github.com/facebook/duckling/pull/106

Reviewed By: blandinw

Differential Revision: D6312605

Pulled By: patapizza

fbshipit-source-id: 69ec673f95ec8a2d86ec207a6d75cd8ebfcdb4f6
2017-11-14 21:00:28 -08:00
Panagiotis Vekris
536f2844e3 Support Greek ordinals
Summary: Adding support for Greek ordinals

Reviewed By: patapizza

Differential Revision: D6263781

fbshipit-source-id: ff339ee51e4e8ad6b0c8f3fa75f5652391dbe48e
2017-11-08 11:00:31 -08:00
Igor Drozdov
5d2c5c78ba Added Distance and Volume Dimensions for Russian language
Summary:
- Added Distance Dimension for Russian language (RU)
- Added Volume Dimension for Russian language (RU)
- Extended `Duckling.Distance.Types.Unit` type definition by adding `Millimetre` representation
Closes https://github.com/facebook/duckling/pull/101

Reviewed By: JonCoens

Differential Revision: D6254070

Pulled By: patapizza

fbshipit-source-id: 579f7a259f76ff1c23ccfe2371afea385eb56aa1
2017-11-08 11:00:31 -08:00
Panagiotis Vekris
e8937e1cd6 Support for Greek durations
Summary: Adding support for Greek time grains and durations.

Reviewed By: patapizza

Differential Revision: D6249955

fbshipit-source-id: 1c69e26
2017-11-06 18:49:36 -08:00
Panagiotis Vekris
fda8c7c759 Support Greek numerals
Summary:
- Setup Greek language (EL)
- Added Greek Numerals

Reviewed By: patapizza

Differential Revision: D6217873

fbshipit-source-id: 379170f
2017-11-02 17:16:18 -07:00
Sam Pepose
251028e691 Uses correct date format for all EN locales
Summary: The date format changes between EN locales (https://en.wikipedia.org/wiki/Date_format_by_country). This diff fixes how dates are handled in each locale.

Reviewed By: patapizza

Differential Revision: D6156147

fbshipit-source-id: 22f296c
2017-10-26 08:49:21 -07:00
Matthijs Mullender
1ade1935b2 Support Dutch dates and times
Summary: [Duckling][Time][NL] Support Dutch dates and times

Reviewed By: patapizza

Differential Revision: D6090294

fbshipit-source-id: 54b8729
2017-10-19 14:04:38 -07:00
Abdallatif Sulaiman
18cd2210ac Add Duration Dimension to Arabic Language
Summary: Closes https://github.com/facebookincubator/duckling/pull/94

Reviewed By: blandinw

Differential Revision: D6078221

Pulled By: patapizza

fbshipit-source-id: b653b24
2017-10-17 16:04:28 -07:00
Julien Odent
1ab5f447d2 en_CA + fix Canadian Thanksgiving
Summary:
* `en_CA` locale
* In Canada, Thanksgiving Day is the second Monday of October.
* Black Friday is the same as the US.
* However Canada observes both DDMM and MMDD formats. Defer to later, falling back to US.

Reviewed By: blandinw

Differential Revision: D6058909

fbshipit-source-id: 3d4e05e
2017-10-16 10:04:43 -07:00
Julien Odent
fb1dcaa138 Chinese locales + fix TW National Day
Summary:
* Moving `ruleNationalDay` from `ZH` rules to specific locales: `zh_CN`, `zh_HK`, `zh_MO`
* Fixed National Day for `zh_TW`.

Reviewed By: blandinw

Differential Revision: D6057565

fbshipit-source-id: 8f9f2ab
2017-10-13 17:04:43 -07:00
Matthijs Mullender
33a08bb76b Support Dutch Durations
Summary:
This change adds support for durations in Dutch/Netherlands (NL)
Implemented: TimeGrain/NL, Durations/NL

Reviewed By: patapizza

Differential Revision: D6049404

fbshipit-source-id: 3621cdb
2017-10-13 12:49:30 -07:00
Julien Odent
ab0ad0256e Locales support
Summary:
* Locales support for the library, following `<Lang>_<Region>` with ISO 639-1 code for `<Lang>` and ISO 3166-1 alpha-2 code for `<Region>` (#33)
* `Locale` opaque type (composite of `Lang` and `Region`) with `makeLocale` smart constructor to only allow valid `(Lang, Region)` combinations
* API: `Context`'s `lang` parameter has been replaced by `locale`, with optional `Region` and backward compatibility.
*  `Rules/<Lang>.hs` exposes
  - `langRules`: cross-locale rules for `<Lang>`, from `<Dimension>/<Lang>/Rules.hs`
  - `localeRules`: locale-specific rules, from `<Dimension>/<Lang>/<Region>/Rules.hs`
  - `defaultRules`: `langRules` + specific rules from select locales to ensure backward-compatibility
* Corpus, tests & classifiers
  - 1 classifier per locale, with default classifier (`<Lang>_XX`) when no locale provided (backward-compatible)
  - Default classifiers are built on existing corpus
  - Locale classifiers are built on
  - `<Dimension>/<Lang>/Corpus.hs` exposes a common `corpus` to all locales of `<Lang>`
  - `<Dimension>/<Lang>/<Region>/Corpus.hs` exposes `allExamples`: a list of examples specific to the locale (following `<Dimension>/<Lang>/<Region>/Rules.hs`).
  - Locale classifiers use the language corpus extended with the locale examples as training set.
  - Locale examples need to use the same `Context` (i.e. reference time) as the language corpus.
  - For backward compatibility, `<Dimension>/<Lang>/Corpus.hs` can expose also `defaultCorpus`, which is `corpus` augmented with specific examples. This is controlled by `getDefaultCorpusForLang` in `Duckling.Ranking.Generate`.
  - Tests run against each classifier to make sure runtime works as expected.
* MM/DD (en_US) vs DD/MM (en_GB) example to illustrate

Reviewed By: JonCoens, blandinw

Differential Revision: D6038096

fbshipit-source-id: f29c28d
2017-10-13 08:34:21 -07:00
Stepan Parunashvili
6f774abe38 georgian numeral support
Summary: Introducing Georgian (KA), and the very beginnings of numeral support

Reviewed By: patapizza

Differential Revision: D5757952

fbshipit-source-id: 89d05f8
2017-09-05 12:19:29 -07:00
dubovinszky
60565c15aa HU Time, TimeGrain
Summary: Closes https://github.com/facebookincubator/duckling/pull/83

Reviewed By: blandinw

Differential Revision: D5681515

Pulled By: patapizza

fbshipit-source-id: 918d0a4
2017-08-22 19:34:33 -07:00