Commit Graph

336 Commits

Author SHA1 Message Date
Quan
c166179d89 Use comma "," as decimal separator for VI
Summary:
In Vietnam, we use comma "," as decimal separator instead of dot "."
Closes https://github.com/facebook/duckling/pull/176

Reviewed By: patapizza

Differential Revision: D7601328

Pulled By: panagosg7

fbshipit-source-id: 7f5725006f9054fe45a24757ec2abc36b7aa6605
2018-04-18 23:15:30 -07:00
Aaron Yue
4df76289bc Implement AmountOfMoney
Summary: implement AmountOfMoney rules and corpus for ZH

Reviewed By: patapizza

Differential Revision: D7508507

fbshipit-source-id: 3591b399a9880c5278587979c6576720343cc123
2018-04-17 16:45:32 -07:00
Aaron Yue
babe317723 fix intersect rule to work with negative numbers
Summary:
Numerals that require intersection with negative numbers don't work, since the negative sign gets parsed
before the intersect rules happen. This fixes it by adding a guard in for positive in the intersection rule.

Reviewed By: patapizza

Differential Revision: D7592225

fbshipit-source-id: 2bc9c708cadeea4012c1f3ef487c61a144325f2a
2018-04-13 19:45:40 -07:00
Chinmay Deshmukh
17e1845e12 Time/EN_CA, EN_US: Add "Groundhogs day" in regex for "Groundhog Day"
Summary: Modify regex to add "Groundhogs day" in regex for "Groundhog Day" in US and CA

Reviewed By: patapizza

Differential Revision: D7605031

fbshipit-source-id: 0cbb45d4edde3345437872f2e5b366c39f6a8b54
2018-04-13 12:15:31 -07:00
Julien Odent
de977fe5cc Duration: Move composite duration rule to English
Summary: "and" is language-specific.

Reviewed By: chinmay87

Differential Revision: D7605874

fbshipit-source-id: 59adcee4784baebe3b9eb775b6763bf3bba8ef8d
2018-04-12 15:15:29 -07:00
RIAN DOUGLAS
fd267ee80b Time/EN: Make "may" latent
Summary: Extended method for making months from a pair, to take a 3-tuple, including a flag indicating whether to make the result latent. Implement the previous mkRuleMonths method in terms of this.

Reviewed By: patapizza

Differential Revision: D7491037

fbshipit-source-id: 4d4c41b46dc0390e17b521480f3daf8306f23834
2018-04-12 14:00:29 -07:00
Aaron Yue
fe0807dced Use lambda-case for Rule production
Summary: change Rule production functions to lambda-case

Reviewed By: patapizza

Differential Revision: D7424413

fbshipit-source-id: edac0290d310578f633ff0208434d1eca038ad9c
2018-04-12 10:15:57 -07:00
Aaron Yue
4ef255e577 Generalize and expand digit specifier usage for hanzi
Summary:
generalize chinese digit specifier (十百千万亿) parsing, and add hanzi tests
These digits specifiers can be parsed as (<num><speci>)<num>,
by using the multiplicater value <num><speci>, and a connect function that adds them together
(two cases, skipping digits [which requires a 零 in between], and digits in consecutive locations).
Note that 个 is technically a digit specifier,
but in Chinese it is never used directly as a numeral specifier, and always as a counter.

Reviewed By: zliu41

Differential Revision: D7424249

fbshipit-source-id: 20a85a7df1f908ee9879e92b904178fa26a9a5e5
2018-04-12 10:00:33 -07:00
Chinmay Deshmukh
f355abacc5 Time/EN: time + n years ago/in n years
Summary:
Add support for things like -
"mlk day three years ago"
"mlk day in three years"
"Labor Day one year from now"

"Thanksgiving in one year"
"Thanksgiving in 2013"

"january in a year"
"january 3 years ago"

Reviewed By: patapizza

Differential Revision: D7581591

fbshipit-source-id: 6f37a83c5acbff28672c2e8a958eaf37c679cab3
2018-04-12 01:10:17 -07:00
Julien Odent
c539ba2f55 Time/EN: Tu BiShvat
Summary: Added precomputed Jewish holiday `Tu BiShvat`.

Reviewed By: chinmay87

Differential Revision: D7596389

fbshipit-source-id: ed770dfa0eca982d5d1896dacff6443ee0fa2d30
2018-04-11 20:15:42 -07:00
Julien Odent
443caeb096 Time/EN_AU: Reconciliation Day
Summary:
First Monday on or after May 27th:
https://publicholidays.com.au/reconciliation-day/

Reviewed By: chinmay87

Differential Revision: D7596237

fbshipit-source-id: 5ad4effb30d1e09a988c93fdb85f9dc60d0a1e54
2018-04-11 20:15:42 -07:00
Souvik Ghosh
ced73dcdcb AmountOfMoney NL Support
Summary:
Hello,
I have added new directory for AmountOfMoney NL Support
Added two files
- Duckling/AmountOfMoney/NL/Corpus.hs
- Duckling/AmountOfMoney/NL/Rules.hs

Updated File
- Duckling/Rules/NL.hs

Added test cases
- tests/Duckling/AmountOfMoney/NL/Tests.hs

Updated Test file
- tests/Duckling/AmountOfMoney/Tests.hs

Updated
duckling.cabal

Thanks for the review and the latest merge. Looking forward

Regards
Closes https://github.com/facebook/duckling/pull/173

Reviewed By: JonCoens

Differential Revision: D7592192

Pulled By: patapizza

fbshipit-source-id: 5895c29bf7f1033e4ffd791d5915a16d230e9375
2018-04-11 17:15:30 -07:00
Julien Odent
120f569ed0 Time/EN: Fix "in" + year
Summary: Now that years are latent, let's absorb the "in" and make them not latent.

Reviewed By: zliu41

Differential Revision: D7587599

fbshipit-source-id: 61a19ac389244df491591d78c28f0301f9124439
2018-04-11 11:00:30 -07:00
Lucas Hosseini
528fea9ad4 Make all years latent
Summary: Make all years latent to prevent any four digit number to be considered a year.

Reviewed By: patapizza

Differential Revision: D7559843

fbshipit-source-id: f3a8625dd7d74edfffb13401278d6b4a5110359a
2018-04-11 05:15:31 -07:00
Julien Odent
944e8d73c2 Time/EN: Islamic holidays
Summary:
Islamic holidays from a few pre-computed reference times, based on the Umm al-Qura Calendar of Saudi Arabia.
http://www.staff.science.uu.nl/~gent0113/islam/ummalqura.htm
http://www.staff.science.uu.nl/~gent0113/islam/ummalqura_principal.htm

Reviewed By: chinmay87

Differential Revision: D7578193

fbshipit-source-id: b9e4428ad91058c24495ce27a0ae945b91291d4f
2018-04-10 16:15:26 -07:00
Chinmay Deshmukh
c567abea1c Time/EN: Rule <Day of month> of month
Summary:
Rule to resolve things like:
"27th of next month"
"15th of last month"
"3rd of this month"

Reviewed By: patapizza

Differential Revision: D7543740

fbshipit-source-id: 20bdc194ed7d1466e2ca42c87b73a04ace7c29ab
2018-04-09 16:45:37 -07:00
Danny Su
1c6ebffb46 Time/ES: use mkRuleSeasons
Summary:
- removed ruleSeason, ruleSeason2, ruleSeason3, ruleSeason4
- replaced above with ruleSeasons and use the `mkRuleSeasons` helper

Reviewed By: patapizza

Differential Revision: D7545455

fbshipit-source-id: 57842f52ab3ab954ba31a200942970b6d0bd040e
2018-04-09 12:30:37 -07:00
Oleksii Dykan
e0e2338dd1 Add proper EOY and EOM rules
Summary: Added rules to cover EOY, EOM, BOY, BOM

Reviewed By: zliu41

Differential Revision: D7492089

fbshipit-source-id: fb0c787d709146534980ccc53e240ae88740a553
2018-04-09 08:15:33 -07:00
Julien Odent
18934b81e2 Time/EN: Fix computed past series + Jewish holidays
Summary:
* Fixed computed past series
* Added a few Jewish holidays, pre-computed and built upon.

Reviewed By: chinmay87

Differential Revision: D7511694

fbshipit-source-id: c66a34e6634a9f86101fb2fd3b4a372d75ba5711
2018-04-06 19:15:25 -07:00
Ziyang Liu
c819dcb665 add Discovery Day in EN/CA
Summary: add Discovery Day in EN/CA

Reviewed By: patapizza

Differential Revision: D7527814

fbshipit-source-id: 053e70b59c121e6853f743cd5b9a2b1a4a45e38c
2018-04-06 13:45:50 -07:00
Ziyang Liu
3547fa3472 Add Boss's Day (closest working day to Oct 16) in EN
Summary: Add Boss's Day (closest working day to Oct 16) in EN

Reviewed By: patapizza

Differential Revision: D7512241

fbshipit-source-id: c8a4a14b77677ed34edebfeb48bfce1019bb059d
2018-04-05 18:00:49 -07:00
Rajesh Shenoy
7a76c938da Add distance to SV
Summary: Earlier distance was implemented, but it was not defined

Reviewed By: patapizza

Differential Revision: D7524341

fbshipit-source-id: e1e8342a6eff5a523a0db31bf301870ae27c513e
2018-04-05 16:00:33 -07:00
Aaron Yue
e542ff5b50 Distribute ruleUnitAmount to all languages
Summary:
enforce units are checked in ruleUnitAmount and ruleAmountUnit, and distribute ruleUnitAmount to all
languages, since in some languages this doesn't work (such as for ZH, where value units must succeed amounts)

Reviewed By: patapizza

Differential Revision: D7509840

fbshipit-source-id: 3a79b39721850f9d0289442925fbc02cf69abefb
2018-04-05 10:45:35 -07:00
Aaron Yue
878bcb9277 Show input for ambiguous parse failures
Summary: show the input that an ambiguous parse is failing at

Reviewed By: patapizza

Differential Revision: D7502191

fbshipit-source-id: 9f0fbf8301413d9007236ba5b6af1f4b41c20269
2018-04-04 10:30:32 -07:00
Aaron Yue
3629fbd503 Prompt ambiguous parses in corpus tests
Summary:
During ranking, due to how candidates are ordered, it is completely possible to have multiple correct candidates
have the exact same rank (equal range and exact equal score). In this case `analyze` returns all of them, which gets
misinterpreted as having multiple tokens in output rather than multiple solutions. Checks this case and gives the
correct prompt for ambiguous parses.

Reviewed By: patapizza

Differential Revision: D7489391

fbshipit-source-id: b66947e37bddb3ac6273843dd79b559aff9d0083
2018-04-03 17:00:27 -07:00
Julien Odent
7b25f5c564 Time/EN: Chinese New Year
Summary: Added computed Chinese New Year.

Reviewed By: chinmay87

Differential Revision: D7492868

fbshipit-source-id: 203f6645dd39adb3ae3a0e97d99c79a1e10a3d93
2018-04-03 16:30:27 -07:00
Julien Odent
04f215febb Time/EN: fix Labour Day, add Heroes' Day and National Patriots' Day
Summary:
* fixed Labour Day for most locales
* added Heroes' Day for JM, PH and ZA
* National Patriots' Day for CA

Reviewed By: chinmay87

Differential Revision: D7492156

fbshipit-source-id: 2a99412a55190ffa5a541b47f6d92d0df928fc3b
2018-04-03 16:00:30 -07:00
Julien Odent
9a734b3d6a Time/EN: Orthodox Easter and -based holidays
Summary:
* precomputed Orthodox Easter dates
* added Orthodox Easter-based holidays for English

Reviewed By: chinmay87

Differential Revision: D7490875

fbshipit-source-id: d24210f6ef01dc10dd9f1f84205f180af7402bb4
2018-04-03 16:00:30 -07:00
Julien Odent
59a9784061 Time/EN: Easter-based holidays
Summary:
- added Easter-based holidays
- fixed mother's day for `GB`

Reviewed By: chinmay87

Differential Revision: D7482119

fbshipit-source-id: e28345a7a3d262e52878ffac1e79b7d2cfb0ed98
2018-04-03 12:15:29 -07:00
Ziyang Liu
7ae9e50c6a Add EN support for this|last|next season
Summary: Add EN support for this|last|next season

Reviewed By: patapizza

Differential Revision: D7443814

fbshipit-source-id: 925296f793fbe03bcb67d8a6af3eb6695347eedd
2018-04-03 01:45:26 -07:00
Ziyang Liu
28e88e9390 Add composite durations
Summary: Add composite durations

Reviewed By: patapizza

Differential Revision: D7450056

fbshipit-source-id: 0d571529341e742a60ecc2cd6e2b13ddc9ac5b1e
2018-04-02 19:30:26 -07:00
Chinmay Deshmukh
e7f285cd1d Time: Return Holiday Name in response (Beta)
Summary:
Return the canonical holiday name if the input had a holiday in it and the date resolves to the date of the holiday.

Add `holiday` to `TimeData`.
Set it in the holiday rule.
Make sure it propogates appropriately in `predNth` and `intersect`.
Make sure it doesn't propogate in other rules.
Add `holiday` to `TimeValue` and set it during `resolve`

Reviewed By: patapizza

Differential Revision: D7441387

fbshipit-source-id: d57602fd294dc6149ab044c05065c1b9250cb331
2018-04-02 10:45:32 -07:00
Julien Odent
2aa05971be Time/EN: More periodic holidays
Summary:
* Added periodic holidays for all English locales
* Fixed father's day in Australia and New Zealand

Reviewed By: chinmay87

Differential Revision: D7450426

fbshipit-source-id: 9ddcf7f08e73e2bb501541d9697d64b7e99a8ef1
2018-03-29 17:00:33 -07:00
Julien Odent
4175fde41a Time/EN: More fixed holidays
Summary: added more fixed dates holidays for English locales

Reviewed By: chinmay87

Differential Revision: D7446233

fbshipit-source-id: 87c30c82c7b4ca9a5a0b2b21bff50d9288ad15e3
2018-03-29 17:00:33 -07:00
Aaron Yue
a9bd094af2 Remove unused and redundant rule
Summary: remove `ruleLastTuesdayLastJuly`, it does the exact same thing as `ruleLastTime`

Reviewed By: patapizza

Differential Revision: D7445836

fbshipit-source-id: 95667db4ce376223acce0ea36a6a0493946ceea1
2018-03-29 15:45:33 -07:00
Ziyang Liu
0c7edb01a9 Add EN support for last night and late last night
Summary: Add EN support for last night and late last night

Reviewed By: patapizza

Differential Revision: D7369829

fbshipit-source-id: bd1dbecf247e428307adfae3e8e83840f6de77b1
2018-03-27 15:45:41 -07:00
Chinmay Deshmukh
131442abd3 Time/EN: "Late Tonight"
Summary:
Add rule for "late tonight" that resolves to 9 pm

Note: Will handle "later tonight" separately

Reviewed By: patapizza

Differential Revision: D7363574

fbshipit-source-id: da26925b7f61424318e4ac99f743138d9ab09e43
2018-03-26 11:15:28 -07:00
Chih-Wei Chang
b0187b27cc Refactor pattern match to hashmap lookup
Summary: Use `HashMap.lookup` to replace the guard match for performance optimization for ordinal rules in Russian.

Reviewed By: chinmay87

Differential Revision: D7372645

fbshipit-source-id: 20555c352a3f65387e0630f5df65c0826af389f1
2018-03-23 12:01:02 -07:00
Oleksii Dykan
5c59b0a214 Add missing month / week / year rules
Summary: Added rules for the beginning / end of year / month / week

Reviewed By: patapizza

Differential Revision: D7356994

fbshipit-source-id: 43489b3709b8ab586384321cf9bc79c45668162a
2018-03-22 11:45:29 -07:00
Wojtek Przechodzeń
15722cc19d two failing tests for day with month
Summary:
Hello,
I wanted to fix problem with two months - May and November for PL language, since it is what I need to use in [wit.ai](http://wit.ai), but.. I'm not sure how to do that. I've created two failing tests, checked `Rules`, but there are already some rules which are responsible for that cases:

`20 listopada` :
`, ( "November" , "listopad|listopada|listopadowi|listopadem|listopadzie|lis\\.?|list\\.?" )`

`20 maja`:
`, ( "May" , "maj|maja|majowi|majem|maju" )`

Can you help me somehow fix that issues because I'm not sure how I can force Duckling to use proper regexp? :)
Closes https://github.com/facebook/duckling/pull/161

Reviewed By: patapizza

Differential Revision: D7353495

Pulled By: panagosg7

fbshipit-source-id: b0c70172e4cc9dbd4635ed64e89ffb463b643f90
2018-03-21 17:00:32 -07:00
Julien Odent
6ee8933139 Distance/SV: fix mil
Summary: 1 mil = 10 km (not 1 km)

Reviewed By: chinmay87

Differential Revision: D7356382

fbshipit-source-id: fc1418216346c3351e1206f1ae59df46a2b8a713
2018-03-21 14:15:29 -07:00
Alex Zveryansky
33ef33787d DDMMYYYY/MMDDYYYY parsing to phone fix
Summary:
"DD MM YYYY", "MM DD YYYY" parsed only as a phone number, but it is also a popular datetime format.
I have added new time patterns and tests for this cases.
Closes https://github.com/facebook/duckling/pull/163

Reviewed By: patapizza

Differential Revision: D7320048

Pulled By: panagosg7

fbshipit-source-id: 5ccf1ccca173126f5d2b9b8b2664181f3b13f78b
2018-03-21 12:45:30 -07:00
Ziyang Liu
22d11c055b Add ZH distance support
Summary: Add ZH distance support

Reviewed By: patapizza

Differential Revision: D7341977

fbshipit-source-id: 301d90515c492da9fa4c4faf5bcc3351eacbed1b
2018-03-21 12:15:28 -07:00
Julien Odent
1a41d3f7f4 Time/EN_US: Washington's birthday/President day + Lincoln's birthday
Summary:
https://en.wikipedia.org/wiki/Washington%27s_Birthday
https://en.wikipedia.org/wiki/Lincoln%27s_Birthday

Reviewed By: chinmay87

Differential Revision: D7344331

fbshipit-source-id: 800dc1b8653efab034afcffc9077e7f67bdf59b8
2018-03-21 11:30:29 -07:00
hend
f70b4ebe07 add distance to SV
Summary: Closes https://github.com/facebook/duckling/pull/168

Reviewed By: chinmay87

Differential Revision: D7322559

Pulled By: patapizza

fbshipit-source-id: 3d4b86068c05a7a82442bc7c182b80b5cf75ec90
2018-03-21 11:15:25 -07:00
Julien Odent
6d7d0ba354 fix build
Summary: `Duckling.Resolve.Options` clashes with `Data.Aeson.Options`.

Reviewed By: niteria

Differential Revision: D7337751

fbshipit-source-id: c9dc633301d45704dbb4975d1942fc5f360a4a44
2018-03-20 10:30:35 -07:00
yasen-yankov
21c3b32e4d Added TimeGrain, Duration, and Ordinal
Summary: Closes https://github.com/facebook/duckling/pull/164

Reviewed By: chinmay87

Differential Revision: D7280110

Pulled By: patapizza

fbshipit-source-id: d98ddd900fe83f06b28afd39ea3311f42716288c
2018-03-19 18:00:29 -07:00
Giri Anantharaman
519c9519a3 Support Tamil numerals
Summary:
* Setup Tamil (TA) language
* Added Numeral Dimension

Reviewed By: patapizza

Differential Revision: D7323636

fbshipit-source-id: 4b1a42197ff4799880cded9ce86b8d7fae1507bc
2018-03-19 16:45:36 -07:00
Chinmay Deshmukh
5ac990bbe2 Return latent entities
Summary: Add an option to return latent time entities. This can be used when one is pretty certain that the input contains a datetime.

Reviewed By: patapizza

Differential Revision: D7254245

fbshipit-source-id: e9e0503cace2691804056fcebdc18fd9090fb181
2018-03-19 14:45:27 -07:00
Julien Odent
490799fe4a Time/EN: Periodic holidays
Summary:
* added periodic holidays for all existing English locales
* moved Independence Day from common to US, as it is country-dependent
* moved Memorial Day from common to US, as it is country-dependent (fixed the one for Canada)
* added Decoration Day as an alias for US Memorial Day

Reviewed By: chinmay87

Differential Revision: D7294129

fbshipit-source-id: 89cdbf91b4c43e1996fa1f4509eac1d1b1978197
2018-03-16 17:17:08 -07:00