Commit Graph

439 Commits

Author SHA1 Message Date
pheaktra21
cd33b26acf Add support for Ordinal, Quantity, & Temperature dimension for KM. (#251)
Summary:
Hello.
I added 3 new dimensions: Ordinal, Quantity, & Temperature. And I also added more data to Numeral dimension.
Thank you!
Pull Request resolved: https://github.com/facebook/duckling/pull/251

Reviewed By: beauby

Differential Revision: D9656646

Pulled By: patapizza

fbshipit-source-id: f17797be811d58b0b5bcd02b83c0a699650c9f6e
2018-09-06 07:01:15 -07:00
kumar-shridhar
4a77765134 Icelandic support
Summary: Pull Request resolved: https://github.com/facebook/duckling/pull/247

Reviewed By: beauby

Differential Revision: D9656660

Pulled By: patapizza

fbshipit-source-id: 993baa9a0e442fc6f6f813a3af717b700c4d89e6
2018-09-06 06:46:02 -07:00
jfulse
b9d39f723e Norwegian numeral improvements (#243)
Summary:
Some small improvements to Norwegian numerals:

- Parse powers of ten with spaces as well as dots: `10 000 == 10.000`
- Some more textual powers of ten: `milliard`, `billion`, ...
- Textual numbers from 21 to 99: `tjueen`, `tjueto`, ...
   (they were already parsed with spaces, as in `tjue en`, but not without)
Pull Request resolved: https://github.com/facebook/duckling/pull/243

Reviewed By: patapizza

Differential Revision: D9505835

Pulled By: haoxuany

fbshipit-source-id: e42a290b3dd2d5e6a82f00514617b4c3c4dd3416
2018-08-31 17:46:08 -07:00
David Magaltadze
ab3b789896 KA AmountOfMoney (#245)
Summary:
Added support for AmountOfMoney for KA
Pull Request resolved: https://github.com/facebook/duckling/pull/245

Differential Revision: D9510854

Pulled By: patapizza

fbshipit-source-id: 966c0f69193dd37ed458339fbc381ca640a3a179
2018-08-28 17:00:50 -07:00
Julien Odent
fd3948169f Time/EN: Intersect "9 tomorrow morning"
Summary:
It didn't intersect before, as `9` is latent and we were guarding against it.
Instead, guard against years (to prevent nonsensical things like `one ninety nine a.m.`) and against 2 latent tokens.

Reviewed By: chinmay87

Differential Revision: D9507044

fbshipit-source-id: 4c97eba8fc1e47bf7b9a78e04dd6b5531898fda6
2018-08-28 16:45:45 -07:00
Nathan Hu
eb0b84a5aa Time/EN: add <datetime> - <datetime> (interval) timezone rule
Summary: Introducing `hasTimezone` flag to prevent double timezone parsing.

Reviewed By: patapizza

Differential Revision: D9317892

fbshipit-source-id: 0e9246abeb65c5fa61725f9b78a1421d85263bdc
2018-08-24 11:45:45 -07:00
Nihar Shah
6a05e40a37 Volume: Intervals Support
Summary:
This diff implements volume intervals in Duckling. It follows the AmountofMoney and Quantity modules in doing so.

I had to make one crucial choice here -- defining the core Volume data type -- and whether the attribute "Unit" was optional (i.e. "Maybe") or not. Like in Quantity and unlike in AmountOfMoney, I made it optional, so that latent volumes can be supported down the line in the codebase.

I also wrote the codebase to be more modular, such that future developers only need to add regular expressions rather than functions for any language. For instance, developers can simply define a fraction (e.g. "eighth") at the start of the file, and new rules will be generated automatically; rather than requiring the developer to create an entirely new rule, as previously. The only (partial) exceptions were in the Arabic and Russian Rules files, where the language structure is more difficult and so I cannot fully implement this. Developers for those two languages may need to write new rules, as before.

Reviewed By: patapizza

Differential Revision: D9043117

fbshipit-source-id: f08de4f167596b5b32d12a79268b8ab92c099b22
2018-08-24 11:31:08 -07:00
Kay Hu
fc801ad600 Time/EN: Holiday name alias fixes
Summary:
Adding holiday aliases:
Veteran Day
Mardi Gras
St. Paddy's Day

Reviewed By: chinmay87

Differential Revision: D9483120

fbshipit-source-id: 50bf8338bbaf5c4a59210331dac00508141be1ee
2018-08-23 17:45:42 -07:00
David Magaltadze
010146d2f4 GE + Duration, Ordinal, TimeGrain, Time (#221)
Summary:
Added initial support for ka_GE for

- Duration
- Ordinal
- TimeGrain
- Time
Pull Request resolved: https://github.com/facebook/duckling/pull/221

Differential Revision: D8840919

Pulled By: haoxuany

fbshipit-source-id: 80f94fbaf25786b0f58cd6598c29cd3663858809
2018-08-22 18:30:48 -07:00
Chinmay Deshmukh
dd60c1e013 Time/EN: in <duration> at <time-of-day>
Summary: Make rule for in <duration> at <time-of-day>

Reviewed By: patapizza

Differential Revision: D9375588

fbshipit-source-id: e0ebea53005b09e0ef8b70bf4f68d73b0f75df08
2018-08-20 00:30:52 -07:00
Gerben Janssen van Doorn
f70f991b38 Added support Lao numerals
Summary: Duckling didn't support Lao numerals, this diff adds it.

Reviewed By: patapizza

Differential Revision: D9323242

fbshipit-source-id: 5dad0d4dfb4843281a327947690e664c91ab8f1a
2018-08-17 10:31:17 -07:00
Daniel Sainati
3910c68262 AmountOfMoney/EN: added support for subunits of dollars
Summary: Added support for recognizing nickels, dimes and quarters, as well as numbers of coins.

Reviewed By: patapizza

Differential Revision: D9317706

fbshipit-source-id: 3fb3d63546c9cfe13c094703ddb583c6ac46585b
2018-08-16 15:45:43 -07:00
Julien Odent
f8c967c28d ExampleMain: show how to query for custom dimensions
Summary:
After including a custom dimension in `duckling.cabal` and importing
the constructor in `ExampleMain.hs`, you can query it using the `dims`
parameters (e.g. `dims=["my-dimension"]`).
Note that a custom dimension file should only expose the constructor (i.e.
`MyDimension(..)`) (and maybe a `corpus`).

https://github.com/facebook/duckling/issues/236

Reviewed By: adanielescu

Differential Revision: D9315608

fbshipit-source-id: 4908e7e27b31201c440a0c0db4cd3d567bd1e6f8
2018-08-16 15:30:43 -07:00
Christian Johns
84a4f30f5a Flip month-match expressions in VI ng(à)y regex (#228)
Summary:
Fix issues where date time matching on double-digit months fails for VI.

Reverse the order of month digit matching in two VI rules. Reversing will attempt to capture a two digit month beginning with 1, then fall back to single digit months optionally preceeded with a 0.

The current regex greedily captures months starting with 1, but only of a single digit. (10, 11, and 12) are left out of this calculation, and thus regexes for capturing ng(à)y fail. This leads us into bad capturing on unexpected rules. When matched properly, the idea rules for matching date times are used.

Resolves #225.

I've added to the VI Time Corpus the case in question in #225 and verified its failure at `master` and success on this branch.
Pull Request resolved: https://github.com/facebook/duckling/pull/228

Reviewed By: chinmay87

Differential Revision: D9303795

Pulled By: patapizza

fbshipit-source-id: aa114f886ab0fdcd95f87730d36e5a686afe2f9c
2018-08-16 13:31:07 -07:00
Julien Odent
546fa113a6 Time: Add "today" helper + reuse "tt" helper
Summary: Instead of `cycleNth TG.Day 0` and `Just . Token Time`.

Reviewed By: chinmay87

Differential Revision: D9344258

fbshipit-source-id: 1731c19f0d83640c5716757c70fcefea4655172f
2018-08-15 12:45:48 -07:00
Julien Odent
a3e87589a1 Time: Reuse "now" helper
Summary: Instead of `cycleNth TG.Second 0`.

Reviewed By: chinmay87

Differential Revision: D9341237

fbshipit-source-id: 153dba6edb0b341afad1ee6f7fa04361415be211
2018-08-15 12:45:48 -07:00
Julien Odent
7fba0f0f28 Time: Fix some intervals for time-of-days
Summary:
"from 1:30 to 2pm" resolved to [1:30, 3pm[ with grain `Minute`.
It now resolves to [1:30, 2:01pm[ (same grain).

Reviewed By: chinmay87

Differential Revision: D9338769

fbshipit-source-id: b76c8cc8ac4b6227825c3d21431261f9fb552509
2018-08-15 12:45:48 -07:00
Julien Odent
53d124ffe8 Time/EN: make upper interval consistent for durations
Summary:
Upper interval bounds should be exclusive. For consistency, fixing this in a few rules related to durations.

https://github.com/facebook/duckling/issues/230
https://github.com/wit-ai/node-wit/issues/178

Reviewed By: chinmay87

Differential Revision: D9332993

fbshipit-source-id: ceddb193c019602c714fceb155baf4c56b77572d
2018-08-15 12:45:48 -07:00
Chinmay Deshmukh
179cec14fb Time/EN Support "Martin Luther Kings Day"
Summary: Expand regex to support "Martin Luther Kings Day"

Reviewed By: patapizza

Differential Revision: D9329681

fbshipit-source-id: 78d184f47daef582b21f5c9511b1972c762c27bc
2018-08-15 12:31:30 -07:00
Julien Odent
af480b80f0 Time/EN: Add Purim and Shushan Purim (Jewish holidays)
Summary:
Following other Jewish holidays convention, starting on sunset of
previous day.

Reviewed By: chinmay87

Differential Revision: D9315490

fbshipit-source-id: bdfef5ba0c7bca6601bb7557d30840b47734b819
2018-08-15 12:01:07 -07:00
Julien Odent
e818b4ce9e Time/EN: Restrict "on" absorption to days
Summary:
We were absorbing "on" followed by any Time token.
Restricting to days of weeks and days of months only.

https://github.com/facebook/duckling/issues/217

Reviewed By: chinmay87

Differential Revision: D9313306

fbshipit-source-id: a0c33658855af8f69631d4d1879cc27062c26a69
2018-08-15 01:16:12 -07:00
Sophia Zheng
3c6d3b38db Add daylight saving start and end times.
Summary:
Added "holidays" for daylight savings start and end dates.
In U.S., Daylight Saving start day is the second Sunday in March,  end day is the first Sunday in November.

Reviewed By: chinmay87

Differential Revision: D9277160

fbshipit-source-id: 71038d42d14c4a241cfbaf9ac4499c44139d2ba6
2018-08-14 11:45:55 -07:00
Julien Odent
b45e1c09f9 Time/EN: Allow for "Chinese New Years"
Summary: as titled

Reviewed By: chinmay87

Differential Revision: D9295927

fbshipit-source-id: d90f5183c0b4d7b483f5cf1bf8957f514954944c
2018-08-14 01:31:13 -07:00
Ziyang Liu
a365dec52a Update custom dimension example to include pattern match on dimension data
Summary: Pull Request resolved: https://github.com/facebook/duckling/pull/233

Reviewed By: watashi

Differential Revision: D9021889

Pulled By: chinmay87

fbshipit-source-id: 1e134cf6d20006e1ee8824fa2fcbb8f7cdc4f197
2018-08-13 16:55:07 -07:00
Daniel Sainati
3de16f3fae added support for Vaisakhi 2000-2030
Summary: Added support for Vaisakhi holiday

Reviewed By: chinmay87

Differential Revision: D9275656

fbshipit-source-id: 21744a93c191837e990ddd381ea46eb7f85b86cf
2018-08-10 15:15:51 -07:00
PhalPheaktra Chhaya
b541354c31 Add Numeral dimension for new language KM. (#234)
Summary:
Hello,
I am new to Haskell, but I would like to add Khmer language (KM) to Duckling.
I have tried to extended Duckling by adding Numeral dimension for new language KM.
Please have a look at it and see what we can improve.

Thanks!
Pull Request resolved: https://github.com/facebook/duckling/pull/234

Reviewed By: blandinw

Differential Revision: D9032639

Pulled By: chinmay87

fbshipit-source-id: 7db19edf732fe6500629cc89e18e0655d7bbc48b
2018-08-03 14:45:55 -07:00
Nathan Hu
945154a174 add (timezone) rule
Summary:
add rule to match
<time> + (timezone)
It needs to be a seperate rule from the existing ruleTimezone because we need to make sure a match on both "(" and ")" before and after the timezone string.

if I do regex "(\b|\()XXX(\b|\))" it can't guarantee the match on both.

Reviewed By: xhavokx

Differential Revision: D9143475

fbshipit-source-id: bda686763c3cf69b9e2096ce05878ba592076814
2018-08-03 12:01:03 -07:00
Ben Powell
984769244f Time/EN: All week, Rest of the week
Summary:
Task also says to add "this week", but this is already covered
by the "this <time>" rule.

Reviewed By: chinmay87

Differential Revision: D9018849

fbshipit-source-id: 210daf87ad4221cf734f260932f7c95c8a3e417a
2018-08-01 09:45:42 -07:00
Chinmay Deshmukh
42ab956c3f Try travis_wait to extend timeout
Summary:
Builds have been timing out with Compiler: : #GHC 8.0.2 C. Try `travis_wait` which extends the timeout to 20 mins instead of 10.

The error has been -
```
Linking dist/build/duckling-test/duckling-test ...
The command "cabal build -j1" exited with 0.
$ cabal test -j1 --show-details=always
Preprocessing library duckling-0.1.6.1...
Preprocessing test suite 'duckling-test' for duckling-0.1.6.1...
Running 1 test suites...
Test suite duckling-test: RUNNING...
No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
```

Reviewed By: blandinw

Differential Revision: D9020909

fbshipit-source-id: 7ccf8d3265860c3336b5191a5e89a493645bba2a
2018-07-26 16:16:13 -07:00
Chinmay Deshmukh
9a4bb48c63 Time: Fix BST and IST offsets
Summary:
Fix the offsets for BST and IST timezones.
BST is +1 (60) and IST is +5:30 (330)

Reviewed By: girifb

Differential Revision: D8981055

fbshipit-source-id: 43ad669c22638031479a7608a8e651b178361c0d
2018-07-25 13:15:50 -07:00
jfulse
e1088c1856 Norwegian improvements (#232)
Summary:
Some small improvements to the Norwegian implementation:

- The written number 8 had a typo: "otte" -> "åtte"
- Add support for half an hour before as e.g. "halv to"
- Add support for alternative clock denotation "klokka"
- Add support for alternative tomorrow denotation "i morra"
Pull Request resolved: https://github.com/facebook/duckling/pull/232

Reviewed By: girifb

Differential Revision: D8986441

Pulled By: chinmay87

fbshipit-source-id: 286617d30415febe1f0eda4bc7475ca5c9610734
2018-07-25 13:15:50 -07:00
Nathan Hu
8e658b2e27 add rules to time
Summary:
this rule can parse time interval like this:
2015-03-28 17:00:00/2015-03-28 21:00:00

Need to check both timeData has the same Grain to avoid ambiguity like the following:

intersect (2015-03-28 17:00:00/2015-03-28 21:00:00)
-- <datetime>/<datetime> (interval) (2015-03-28 17:00:00/2015-03-28)
-- -- intersect (2015-03-28 17:00:00)
-- -- -- yyyy-mm-dd (2015-03-28)
-- -- -- -- regex (2015-03-28)
-- -- -- hh:mm:ss (17:00:00)
-- -- -- -- regex (17:00:00)
-- -- regex (/)
-- -- yyyy-mm-dd (2015-03-28)
-- -- -- regex (2015-03-28)
-- hh:mm:ss (21:00:00)
-- -- regex (21:00:00)

Reviewed By: xhavokx

Differential Revision: D8951108

fbshipit-source-id: 5d5084c8c7e8d9420b9899ccbb8d757a0322be00
2018-07-23 12:31:49 -07:00
Yujiao Zhou
20c8a3750d parse yyyy-mm
Summary: as titled.

Reviewed By: haoxuany

Differential Revision: D8895617

fbshipit-source-id: 2c0ccdcbb39945d836bd64f5ff9d790c8f636b79
2018-07-20 10:30:46 -07:00
Nathan Hu
98463ae14d update typeclass to make cast work.
Summary: update typeclass to make cast work.

Reviewed By: watashi

Differential Revision: D8918769

fbshipit-source-id: af44737d439d8748926114ca7ff606fcda848946
2018-07-19 12:01:42 -07:00
Nivin Lawrence
e279c5642e added tamil ordinal
Summary: support tamil ordinal

Reviewed By: girifb

Differential Revision: D8713963

fbshipit-source-id: c0760f0cf13d983473d29e18425ba926584c3072
2018-07-16 17:15:53 -07:00
Julien Odent
9c367ab6cd Don't accept dashes (-) as token separators
Summary:
This is causing some issues, e.g. `20-30` resolving to 8:30pm (latent).
Updating `Numeral` rules to account for that (`EN`, `FR`, `NL` following tests).

Differential Revision: D8854891

fbshipit-source-id: ba17099b014d9cf2f48a7d85147cc890b02578f5
2018-07-16 06:00:40 -07:00
Arunavha Chanda
d5555d0149 Numeral/BN: Adding Bengali numeral support to Duckling
Summary: Added support for Bengali numerals

Reviewed By: patapizza

Differential Revision: D8730468

fbshipit-source-id: dc36017e24d796f35abc477a0b8b317218c64a6a
2018-07-09 12:30:30 -07:00
Cuong Dinh Tri Nguyen
4e11613d39 Supported AmountOfMoney intervals and improved Numeral
Summary:
Support amount of money intervals for VI

Modify ruleNg, ruleDollar, and ruleVND to better capture the usage of VI

In Numeral, add "ngàn" - a common synonym of "nghìn", and "chục" - colloquially used to count tens. Remove "?" in regex of other words as it does not make sense.

Reviewed By: haoxuany

Differential Revision: D8734066

fbshipit-source-id: 15f879ab796025882c85f0ce9f1677c501b364a0
2018-07-09 10:00:42 -07:00
Nathan Hu
9d757ea449 new rule to handle 13/APR/2018
Summary:
Adding new rule to support:
"31/Oct/1974"
"31st Oct 1974"

Reviewed By: patapizza

Differential Revision: D8752684

fbshipit-source-id: 06b29ffde0d41a6133489a882dc3259570fe6a5c
2018-07-06 14:46:03 -07:00
Tiago Baptista
11e43c0775 Time/EN: Add support to YYYYQQ and YYQQ expressions (#219)
Summary:
Add support to expressions like:
- 2018Q4
- 18Q4
Closes https://github.com/facebook/duckling/pull/219

Reviewed By: haoxuany

Differential Revision: D8713864

Pulled By: patapizza

fbshipit-source-id: 491dfd408c6c2d6afdb1d70f738f87af0456bf40
2018-07-02 15:15:38 -07:00
Julien Odent
f5bd1efb4a Time/EN: fix Palm Sunday regex
Summary:
* Fixes https://github.com/facebook/duckling/issues/215.
* `regenLangClassifiers` helper

Reviewed By: chinmay87

Differential Revision: D8663109

fbshipit-source-id: 5cbbc05729caabadeb0a88190ce00312174a4fdb
2018-06-27 12:30:42 -07:00
Julien Odent
ead526a820 Time/FR: fix month intervals
Summary:
* Fixes https://github.com/facebook/duckling/issues/213
* Updated month interval rules to handle ordinals and spelled out numerals (e.g. "du premier au quinze juin")

Reviewed By: l5t

Differential Revision: D8615867

fbshipit-source-id: fac7c4aa14e175bc3f9cc7c672008a6bc6063055
2018-06-26 11:45:42 -07:00
Tero Laxström
4ed1ed83ed Basics for Finnish (#210)
Summary:
Adds Locale and Numeral for Finnish
Closes https://github.com/facebook/duckling/pull/210

Reviewed By: JonCoens

Differential Revision: D8430386

Pulled By: patapizza

fbshipit-source-id: a3c8b3b3419b7f43e2ef332cdb1fb8fc07da3bec
2018-06-19 10:45:27 -07:00
David Magaltadze
713e5db9d6 Added support of Numeral for ka_GE (#211)
Summary:
* `ka_GE` locale
* full support for Numeral
Closes https://github.com/facebook/duckling/pull/211

Reviewed By: chinmay87

Differential Revision: D8456241

Pulled By: patapizza

fbshipit-source-id: 35fb432191bd20a4965503efde94c5ea29d0c50d
2018-06-18 13:31:14 -07:00
Lucas Pasqualin
7d469a2f85 adds support for during month and regenerates classifiers (#208)
Summary:
Adds support for catching `during <month>`.

Wasn't sure whether or not to commit the classifiers, let me know if I got this right :)

Thanks!
Closes https://github.com/facebook/duckling/pull/208

Reviewed By: chinmay87

Differential Revision: D8427153

Pulled By: patapizza

fbshipit-source-id: 72c9567939cd0eeab62870e48d6e084cfe290bf1
2018-06-14 13:15:42 -07:00
Anshuman Chhabra
5ab49060e1 Added Temperature support for Hindi (HI) language (#206)
Summary:
This PR adds Temperature dimension support for Hindi (HI). Let me know if there are any changes to be made :)

Cheers!
Closes https://github.com/facebook/duckling/pull/206

Reviewed By: haoxuany

Differential Revision: D8381472

Pulled By: patapizza

fbshipit-source-id: a085ba73910a80a51385f01475d9ba75f72fd4c5
2018-06-13 17:00:35 -07:00
Julien Odent
c7bef34bc2 Duration/EN: Fix composite durations without delimiters
Summary: It would only work with commas/and-separated tokens.

Reviewed By: JonCoens

Differential Revision: D8381351

fbshipit-source-id: eafceeaf5d41bf60aaaf78c3bd6835a768e2b0b6
2018-06-12 10:45:42 -07:00
Julien Odent
9558b7a766 Time/EN: Fix 'ten thirty' latent
Summary:
* fix latent time of days like "ten thirty"
* don't parse time of days above 12 with meridiem like "13 am"

Reviewed By: haoxuany

Differential Revision: D8363720

fbshipit-source-id: 8236babdd5b8ab4163ad39704a295be048d1b03e
2018-06-11 14:00:26 -07:00
Julien Odent
e8286f762c Numeral/RO: Fix multipliers with values above 20
Summary: In Romanian, for numerals above 20, we say "20 de milioane", not "20 milioane".

Reviewed By: haoxuany

Differential Revision: D8334109

fbshipit-source-id: a7fc83440334ab9b1f0511f315029e28449f9771
2018-06-11 11:00:38 -07:00
Aaron Yue
20c9a47f9e bump duckling version
Summary: bump to v0.1.6.1

Reviewed By: patapizza

Differential Revision: D8343952

fbshipit-source-id: de76f3e24281bcb6f8bce6b34ab1a873e5766ba9
2018-06-09 11:30:36 -07:00