Commit Graph

750 Commits

Author SHA1 Message Date
Chinmay Deshmukh
9077f9e147 AmountOfMoney,Numeral/EN: Add support for lakh and crore
Summary:
Add support for lakh (1e5) and crore (1e7) in Numeral.

This will help resolve "9-7 lakh rupees" to (Rs. 9, Rs. 700000). Next step is to resolve it to (Rs. 900000, Rs. 700000)

Reviewed By: patapizza

Differential Revision: D10417678

fbshipit-source-id: 670115f22015698ff8c7a9d96afd624efd4c864e
2018-10-17 10:30:50 -07:00
Colin Clemmons
ea1ed47336 Time/EN: Adds Support for Parsi New Year
Summary: Adds support for Parsi New Year

Reviewed By: patapizza

Differential Revision: D10208163

fbshipit-source-id: 6c125ab4310cb81c239d9a2ee3c765a4a1841a5c
2018-10-16 13:16:52 -07:00
Makan Diarra
4943798c75 Extend support for Ramadan and Eid al-Fitr from 2029 to 2050
Summary: Extended Ramadan and Eid al-Fitr from 2029 to 2050 in computed.hs

Reviewed By: chinmay87

Differential Revision: D10304284

fbshipit-source-id: 5240046977632411bf067769499e6afbcda0af7b
2018-10-12 10:01:09 -07:00
Egxona Ferati
f2fd05f9c2 Add Guru Gobind Singh Jayanti holiday
Summary: Adding support for Guru Gobind Singh Jayanti holiday from 2000 to 2030.

Reviewed By: chinmay87

Differential Revision: D10148514

fbshipit-source-id: 79e35bcca37f7a87fc6b46fda41edf0a57b424a6
2018-10-03 15:30:52 -07:00
Akhil Ravidas
e349942541 Support malayalam ordinals
Summary: Add support for malayalam ordinals.

Reviewed By: patapizza

Differential Revision: D10097075

fbshipit-source-id: 90be5f05d1f9cf4cc6558a4583c8c72518fe0ab0
2018-09-28 10:30:54 -07:00
Akhil Ravidas
c46dbb0d92 Support malayalam numerals
Summary: Add support for malayalam numerals in duckling.

Reviewed By: patapizza

Differential Revision: D10075358

fbshipit-source-id: ce7fee2a71d862391980481b018c513981249f3d
2018-09-28 10:30:54 -07:00
Konstantin Kloster
ee24a4196d small bugfix in german numeral dimension (#257)
Summary:
There is a small bug in the German numeral dimension. The number "fünfzehn"(15) cannot be recognized correctly due to a typo. This pull request should fix it:)
Pull Request resolved: https://github.com/facebook/duckling/pull/257

Reviewed By: girifb

Differential Revision: D9810012

Pulled By: patapizza

fbshipit-source-id: e2408a2dc237590b24840260e2a97e9b5c8231f3
2018-09-13 14:30:44 -07:00
Albert Morgese
2ace5d474b AmountOfMoney/EN: Add basic latent support
Summary: Implemented basic latent value support for AmountOfMoney (along with a corpus).  Currently only supports raw numbers.

Reviewed By: chinmay87

Differential Revision: D9777183

fbshipit-source-id: 16a454d451b74ddc526e0db1ddc9b4959b677ca6
2018-09-12 15:30:58 -07:00
Michelle Prawiro
5d8fb7e7fe AmountOfMoney/ID: Support intervals
Summary: Adding interval support for `AmountOfMoney` in Indonesian.

Reviewed By: adanielescu

Differential Revision: D9416842

fbshipit-source-id: 13f1ed74e6ef1e5c12a2f5d785e44069eefe555b
2018-09-07 10:45:45 -07:00
jfulse
d5639056f3 Norwegian currencies (#244)
Summary:
Some small improvements to Norwegian currencies:

- Parse more currencies. Some were not yet defined in `AmountOfMoney.Types` and thus were added
- Parse `norske kroner` as well as `kroner`
- Parse cents of more currencies
- Remove duplicate rule `ruleIntersectXCentsWithAnd` (duplicated in `ruleIntersectAndXCents`)
Pull Request resolved: https://github.com/facebook/duckling/pull/244

Reviewed By: patapizza

Differential Revision: D9505839

Pulled By: haoxuany

fbshipit-source-id: 69c777a24e5138c04f3ada56c5d8541c9837bb8c
2018-09-06 21:00:57 -07:00
Julien Odent
d7eced7e33 Up aeson under 1.5
Summary: Closes https://github.com/facebook/duckling/issues/240.

Reviewed By: beauby

Differential Revision: D9682452

fbshipit-source-id: 15d3c72fe2240880d97971842cfec95cae0aff2b
2018-09-06 08:00:49 -07:00
pheaktra21
cd33b26acf Add support for Ordinal, Quantity, & Temperature dimension for KM. (#251)
Summary:
Hello.
I added 3 new dimensions: Ordinal, Quantity, & Temperature. And I also added more data to Numeral dimension.
Thank you!
Pull Request resolved: https://github.com/facebook/duckling/pull/251

Reviewed By: beauby

Differential Revision: D9656646

Pulled By: patapizza

fbshipit-source-id: f17797be811d58b0b5bcd02b83c0a699650c9f6e
2018-09-06 07:01:15 -07:00
kumar-shridhar
4a77765134 Icelandic support
Summary: Pull Request resolved: https://github.com/facebook/duckling/pull/247

Reviewed By: beauby

Differential Revision: D9656660

Pulled By: patapizza

fbshipit-source-id: 993baa9a0e442fc6f6f813a3af717b700c4d89e6
2018-09-06 06:46:02 -07:00
jfulse
b9d39f723e Norwegian numeral improvements (#243)
Summary:
Some small improvements to Norwegian numerals:

- Parse powers of ten with spaces as well as dots: `10 000 == 10.000`
- Some more textual powers of ten: `milliard`, `billion`, ...
- Textual numbers from 21 to 99: `tjueen`, `tjueto`, ...
   (they were already parsed with spaces, as in `tjue en`, but not without)
Pull Request resolved: https://github.com/facebook/duckling/pull/243

Reviewed By: patapizza

Differential Revision: D9505835

Pulled By: haoxuany

fbshipit-source-id: e42a290b3dd2d5e6a82f00514617b4c3c4dd3416
2018-08-31 17:46:08 -07:00
David Magaltadze
ab3b789896 KA AmountOfMoney (#245)
Summary:
Added support for AmountOfMoney for KA
Pull Request resolved: https://github.com/facebook/duckling/pull/245

Differential Revision: D9510854

Pulled By: patapizza

fbshipit-source-id: 966c0f69193dd37ed458339fbc381ca640a3a179
2018-08-28 17:00:50 -07:00
Julien Odent
fd3948169f Time/EN: Intersect "9 tomorrow morning"
Summary:
It didn't intersect before, as `9` is latent and we were guarding against it.
Instead, guard against years (to prevent nonsensical things like `one ninety nine a.m.`) and against 2 latent tokens.

Reviewed By: chinmay87

Differential Revision: D9507044

fbshipit-source-id: 4c97eba8fc1e47bf7b9a78e04dd6b5531898fda6
2018-08-28 16:45:45 -07:00
Nathan Hu
eb0b84a5aa Time/EN: add <datetime> - <datetime> (interval) timezone rule
Summary: Introducing `hasTimezone` flag to prevent double timezone parsing.

Reviewed By: patapizza

Differential Revision: D9317892

fbshipit-source-id: 0e9246abeb65c5fa61725f9b78a1421d85263bdc
2018-08-24 11:45:45 -07:00
Nihar Shah
6a05e40a37 Volume: Intervals Support
Summary:
This diff implements volume intervals in Duckling. It follows the AmountofMoney and Quantity modules in doing so.

I had to make one crucial choice here -- defining the core Volume data type -- and whether the attribute "Unit" was optional (i.e. "Maybe") or not. Like in Quantity and unlike in AmountOfMoney, I made it optional, so that latent volumes can be supported down the line in the codebase.

I also wrote the codebase to be more modular, such that future developers only need to add regular expressions rather than functions for any language. For instance, developers can simply define a fraction (e.g. "eighth") at the start of the file, and new rules will be generated automatically; rather than requiring the developer to create an entirely new rule, as previously. The only (partial) exceptions were in the Arabic and Russian Rules files, where the language structure is more difficult and so I cannot fully implement this. Developers for those two languages may need to write new rules, as before.

Reviewed By: patapizza

Differential Revision: D9043117

fbshipit-source-id: f08de4f167596b5b32d12a79268b8ab92c099b22
2018-08-24 11:31:08 -07:00
Kay Hu
fc801ad600 Time/EN: Holiday name alias fixes
Summary:
Adding holiday aliases:
Veteran Day
Mardi Gras
St. Paddy's Day

Reviewed By: chinmay87

Differential Revision: D9483120

fbshipit-source-id: 50bf8338bbaf5c4a59210331dac00508141be1ee
2018-08-23 17:45:42 -07:00
David Magaltadze
010146d2f4 GE + Duration, Ordinal, TimeGrain, Time (#221)
Summary:
Added initial support for ka_GE for

- Duration
- Ordinal
- TimeGrain
- Time
Pull Request resolved: https://github.com/facebook/duckling/pull/221

Differential Revision: D8840919

Pulled By: haoxuany

fbshipit-source-id: 80f94fbaf25786b0f58cd6598c29cd3663858809
2018-08-22 18:30:48 -07:00
Chinmay Deshmukh
dd60c1e013 Time/EN: in <duration> at <time-of-day>
Summary: Make rule for in <duration> at <time-of-day>

Reviewed By: patapizza

Differential Revision: D9375588

fbshipit-source-id: e0ebea53005b09e0ef8b70bf4f68d73b0f75df08
2018-08-20 00:30:52 -07:00
Gerben Janssen van Doorn
f70f991b38 Added support Lao numerals
Summary: Duckling didn't support Lao numerals, this diff adds it.

Reviewed By: patapizza

Differential Revision: D9323242

fbshipit-source-id: 5dad0d4dfb4843281a327947690e664c91ab8f1a
2018-08-17 10:31:17 -07:00
Daniel Sainati
3910c68262 AmountOfMoney/EN: added support for subunits of dollars
Summary: Added support for recognizing nickels, dimes and quarters, as well as numbers of coins.

Reviewed By: patapizza

Differential Revision: D9317706

fbshipit-source-id: 3fb3d63546c9cfe13c094703ddb583c6ac46585b
2018-08-16 15:45:43 -07:00
Julien Odent
f8c967c28d ExampleMain: show how to query for custom dimensions
Summary:
After including a custom dimension in `duckling.cabal` and importing
the constructor in `ExampleMain.hs`, you can query it using the `dims`
parameters (e.g. `dims=["my-dimension"]`).
Note that a custom dimension file should only expose the constructor (i.e.
`MyDimension(..)`) (and maybe a `corpus`).

https://github.com/facebook/duckling/issues/236

Reviewed By: adanielescu

Differential Revision: D9315608

fbshipit-source-id: 4908e7e27b31201c440a0c0db4cd3d567bd1e6f8
2018-08-16 15:30:43 -07:00
Christian Johns
84a4f30f5a Flip month-match expressions in VI ng(à)y regex (#228)
Summary:
Fix issues where date time matching on double-digit months fails for VI.

Reverse the order of month digit matching in two VI rules. Reversing will attempt to capture a two digit month beginning with 1, then fall back to single digit months optionally preceeded with a 0.

The current regex greedily captures months starting with 1, but only of a single digit. (10, 11, and 12) are left out of this calculation, and thus regexes for capturing ng(à)y fail. This leads us into bad capturing on unexpected rules. When matched properly, the idea rules for matching date times are used.

Resolves #225.

I've added to the VI Time Corpus the case in question in #225 and verified its failure at `master` and success on this branch.
Pull Request resolved: https://github.com/facebook/duckling/pull/228

Reviewed By: chinmay87

Differential Revision: D9303795

Pulled By: patapizza

fbshipit-source-id: aa114f886ab0fdcd95f87730d36e5a686afe2f9c
2018-08-16 13:31:07 -07:00
Julien Odent
546fa113a6 Time: Add "today" helper + reuse "tt" helper
Summary: Instead of `cycleNth TG.Day 0` and `Just . Token Time`.

Reviewed By: chinmay87

Differential Revision: D9344258

fbshipit-source-id: 1731c19f0d83640c5716757c70fcefea4655172f
2018-08-15 12:45:48 -07:00
Julien Odent
a3e87589a1 Time: Reuse "now" helper
Summary: Instead of `cycleNth TG.Second 0`.

Reviewed By: chinmay87

Differential Revision: D9341237

fbshipit-source-id: 153dba6edb0b341afad1ee6f7fa04361415be211
2018-08-15 12:45:48 -07:00
Julien Odent
7fba0f0f28 Time: Fix some intervals for time-of-days
Summary:
"from 1:30 to 2pm" resolved to [1:30, 3pm[ with grain `Minute`.
It now resolves to [1:30, 2:01pm[ (same grain).

Reviewed By: chinmay87

Differential Revision: D9338769

fbshipit-source-id: b76c8cc8ac4b6227825c3d21431261f9fb552509
2018-08-15 12:45:48 -07:00
Julien Odent
53d124ffe8 Time/EN: make upper interval consistent for durations
Summary:
Upper interval bounds should be exclusive. For consistency, fixing this in a few rules related to durations.

https://github.com/facebook/duckling/issues/230
https://github.com/wit-ai/node-wit/issues/178

Reviewed By: chinmay87

Differential Revision: D9332993

fbshipit-source-id: ceddb193c019602c714fceb155baf4c56b77572d
2018-08-15 12:45:48 -07:00
Chinmay Deshmukh
179cec14fb Time/EN Support "Martin Luther Kings Day"
Summary: Expand regex to support "Martin Luther Kings Day"

Reviewed By: patapizza

Differential Revision: D9329681

fbshipit-source-id: 78d184f47daef582b21f5c9511b1972c762c27bc
2018-08-15 12:31:30 -07:00
Julien Odent
af480b80f0 Time/EN: Add Purim and Shushan Purim (Jewish holidays)
Summary:
Following other Jewish holidays convention, starting on sunset of
previous day.

Reviewed By: chinmay87

Differential Revision: D9315490

fbshipit-source-id: bdfef5ba0c7bca6601bb7557d30840b47734b819
2018-08-15 12:01:07 -07:00
Julien Odent
e818b4ce9e Time/EN: Restrict "on" absorption to days
Summary:
We were absorbing "on" followed by any Time token.
Restricting to days of weeks and days of months only.

https://github.com/facebook/duckling/issues/217

Reviewed By: chinmay87

Differential Revision: D9313306

fbshipit-source-id: a0c33658855af8f69631d4d1879cc27062c26a69
2018-08-15 01:16:12 -07:00
Sophia Zheng
3c6d3b38db Add daylight saving start and end times.
Summary:
Added "holidays" for daylight savings start and end dates.
In U.S., Daylight Saving start day is the second Sunday in March,  end day is the first Sunday in November.

Reviewed By: chinmay87

Differential Revision: D9277160

fbshipit-source-id: 71038d42d14c4a241cfbaf9ac4499c44139d2ba6
2018-08-14 11:45:55 -07:00
Julien Odent
b45e1c09f9 Time/EN: Allow for "Chinese New Years"
Summary: as titled

Reviewed By: chinmay87

Differential Revision: D9295927

fbshipit-source-id: d90f5183c0b4d7b483f5cf1bf8957f514954944c
2018-08-14 01:31:13 -07:00
Ziyang Liu
a365dec52a Update custom dimension example to include pattern match on dimension data
Summary: Pull Request resolved: https://github.com/facebook/duckling/pull/233

Reviewed By: watashi

Differential Revision: D9021889

Pulled By: chinmay87

fbshipit-source-id: 1e134cf6d20006e1ee8824fa2fcbb8f7cdc4f197
2018-08-13 16:55:07 -07:00
Daniel Sainati
3de16f3fae added support for Vaisakhi 2000-2030
Summary: Added support for Vaisakhi holiday

Reviewed By: chinmay87

Differential Revision: D9275656

fbshipit-source-id: 21744a93c191837e990ddd381ea46eb7f85b86cf
2018-08-10 15:15:51 -07:00
PhalPheaktra Chhaya
b541354c31 Add Numeral dimension for new language KM. (#234)
Summary:
Hello,
I am new to Haskell, but I would like to add Khmer language (KM) to Duckling.
I have tried to extended Duckling by adding Numeral dimension for new language KM.
Please have a look at it and see what we can improve.

Thanks!
Pull Request resolved: https://github.com/facebook/duckling/pull/234

Reviewed By: blandinw

Differential Revision: D9032639

Pulled By: chinmay87

fbshipit-source-id: 7db19edf732fe6500629cc89e18e0655d7bbc48b
2018-08-03 14:45:55 -07:00
Nathan Hu
945154a174 add (timezone) rule
Summary:
add rule to match
<time> + (timezone)
It needs to be a seperate rule from the existing ruleTimezone because we need to make sure a match on both "(" and ")" before and after the timezone string.

if I do regex "(\b|\()XXX(\b|\))" it can't guarantee the match on both.

Reviewed By: xhavokx

Differential Revision: D9143475

fbshipit-source-id: bda686763c3cf69b9e2096ce05878ba592076814
2018-08-03 12:01:03 -07:00
Ben Powell
984769244f Time/EN: All week, Rest of the week
Summary:
Task also says to add "this week", but this is already covered
by the "this <time>" rule.

Reviewed By: chinmay87

Differential Revision: D9018849

fbshipit-source-id: 210daf87ad4221cf734f260932f7c95c8a3e417a
2018-08-01 09:45:42 -07:00
Chinmay Deshmukh
42ab956c3f Try travis_wait to extend timeout
Summary:
Builds have been timing out with Compiler: : #GHC 8.0.2 C. Try `travis_wait` which extends the timeout to 20 mins instead of 10.

The error has been -
```
Linking dist/build/duckling-test/duckling-test ...
The command "cabal build -j1" exited with 0.
$ cabal test -j1 --show-details=always
Preprocessing library duckling-0.1.6.1...
Preprocessing test suite 'duckling-test' for duckling-0.1.6.1...
Running 1 test suites...
Test suite duckling-test: RUNNING...
No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
```

Reviewed By: blandinw

Differential Revision: D9020909

fbshipit-source-id: 7ccf8d3265860c3336b5191a5e89a493645bba2a
2018-07-26 16:16:13 -07:00
Chinmay Deshmukh
9a4bb48c63 Time: Fix BST and IST offsets
Summary:
Fix the offsets for BST and IST timezones.
BST is +1 (60) and IST is +5:30 (330)

Reviewed By: girifb

Differential Revision: D8981055

fbshipit-source-id: 43ad669c22638031479a7608a8e651b178361c0d
2018-07-25 13:15:50 -07:00
jfulse
e1088c1856 Norwegian improvements (#232)
Summary:
Some small improvements to the Norwegian implementation:

- The written number 8 had a typo: "otte" -> "åtte"
- Add support for half an hour before as e.g. "halv to"
- Add support for alternative clock denotation "klokka"
- Add support for alternative tomorrow denotation "i morra"
Pull Request resolved: https://github.com/facebook/duckling/pull/232

Reviewed By: girifb

Differential Revision: D8986441

Pulled By: chinmay87

fbshipit-source-id: 286617d30415febe1f0eda4bc7475ca5c9610734
2018-07-25 13:15:50 -07:00
Nathan Hu
8e658b2e27 add rules to time
Summary:
this rule can parse time interval like this:
2015-03-28 17:00:00/2015-03-28 21:00:00

Need to check both timeData has the same Grain to avoid ambiguity like the following:

intersect (2015-03-28 17:00:00/2015-03-28 21:00:00)
-- <datetime>/<datetime> (interval) (2015-03-28 17:00:00/2015-03-28)
-- -- intersect (2015-03-28 17:00:00)
-- -- -- yyyy-mm-dd (2015-03-28)
-- -- -- -- regex (2015-03-28)
-- -- -- hh:mm:ss (17:00:00)
-- -- -- -- regex (17:00:00)
-- -- regex (/)
-- -- yyyy-mm-dd (2015-03-28)
-- -- -- regex (2015-03-28)
-- hh:mm:ss (21:00:00)
-- -- regex (21:00:00)

Reviewed By: xhavokx

Differential Revision: D8951108

fbshipit-source-id: 5d5084c8c7e8d9420b9899ccbb8d757a0322be00
2018-07-23 12:31:49 -07:00
Yujiao Zhou
20c8a3750d parse yyyy-mm
Summary: as titled.

Reviewed By: haoxuany

Differential Revision: D8895617

fbshipit-source-id: 2c0ccdcbb39945d836bd64f5ff9d790c8f636b79
2018-07-20 10:30:46 -07:00
Nathan Hu
98463ae14d update typeclass to make cast work.
Summary: update typeclass to make cast work.

Reviewed By: watashi

Differential Revision: D8918769

fbshipit-source-id: af44737d439d8748926114ca7ff606fcda848946
2018-07-19 12:01:42 -07:00
Nivin Lawrence
e279c5642e added tamil ordinal
Summary: support tamil ordinal

Reviewed By: girifb

Differential Revision: D8713963

fbshipit-source-id: c0760f0cf13d983473d29e18425ba926584c3072
2018-07-16 17:15:53 -07:00
Julien Odent
9c367ab6cd Don't accept dashes (-) as token separators
Summary:
This is causing some issues, e.g. `20-30` resolving to 8:30pm (latent).
Updating `Numeral` rules to account for that (`EN`, `FR`, `NL` following tests).

Differential Revision: D8854891

fbshipit-source-id: ba17099b014d9cf2f48a7d85147cc890b02578f5
2018-07-16 06:00:40 -07:00
Arunavha Chanda
d5555d0149 Numeral/BN: Adding Bengali numeral support to Duckling
Summary: Added support for Bengali numerals

Reviewed By: patapizza

Differential Revision: D8730468

fbshipit-source-id: dc36017e24d796f35abc477a0b8b317218c64a6a
2018-07-09 12:30:30 -07:00
Cuong Dinh Tri Nguyen
4e11613d39 Supported AmountOfMoney intervals and improved Numeral
Summary:
Support amount of money intervals for VI

Modify ruleNg, ruleDollar, and ruleVND to better capture the usage of VI

In Numeral, add "ngàn" - a common synonym of "nghìn", and "chục" - colloquially used to count tens. Remove "?" in regex of other words as it does not make sense.

Reviewed By: haoxuany

Differential Revision: D8734066

fbshipit-source-id: 15f879ab796025882c85f0ce9f1677c501b364a0
2018-07-09 10:00:42 -07:00
Nathan Hu
9d757ea449 new rule to handle 13/APR/2018
Summary:
Adding new rule to support:
"31/Oct/1974"
"31st Oct 1974"

Reviewed By: patapizza

Differential Revision: D8752684

fbshipit-source-id: 06b29ffde0d41a6133489a882dc3259570fe6a5c
2018-07-06 14:46:03 -07:00