Commit Graph

123 Commits

Author SHA1 Message Date
Hamza Kwisaba
392e584d1b swahili numerical support
Summary: Adding support for swahili numerals

Reviewed By: patapizza

Differential Revision: D13637092

fbshipit-source-id: 6fe0facfa74caae6fd00e0e84b09571aca616f21
2019-01-18 01:16:13 -08:00
Eelco den Heijer
f3088ae252 Added Quantity for NL, small fix for Volume NL (#302)
Summary:
* Added Quantity for NL (kg, mg, g, cup)
* Small fix for Volume (removed Gallon, fixed 'milliliter' typo)
Pull Request resolved: https://github.com/facebook/duckling/pull/302

Reviewed By: chinmay87

Differential Revision: D13554637

Pulled By: patapizza

fbshipit-source-id: 445fcb062c8cc2643b8e7810722759c168b5242f
2019-01-02 14:01:54 -08:00
Michele Riva
9a45a0cf7b Added IT Rules and Corpus for AmountOfMoney (#284)
Summary:
Hi,
I've added Rules and Corpus for Italian language on "AmountOfMoney".
Running `stack test`, I always get `empty result on "10 000 dollari"`.
I've checked my code more than once but I can't figure out what is not working. I made a PR in order to make it easier for you to understand what's going wrong :)

I really hope that my contribution can help.
Regards!
Pull Request resolved: https://github.com/facebook/duckling/pull/284

Reviewed By: chinmay87

Differential Revision: D13551890

Pulled By: patapizza

fbshipit-source-id: c1b448e44c2c6720ad93f61c6396f4d75231131e
2019-01-02 14:01:53 -08:00
uugan
69ffb0f39d Mongolian (#277)
Summary:
MN Setup+Numeral
Pull Request resolved: https://github.com/facebook/duckling/pull/277

Reviewed By: chinmay87

Differential Revision: D13028169

Pulled By: patapizza

fbshipit-source-id: 7d587300918b3ed6211b6ce4147350147b4dc89e
2018-11-26 15:31:08 -08:00
Vitaly Shaydurov
b40e2147a9 Time/UK: added dimensions
Summary:
- added Time dimension for UK language
- added tests for Time dimension for UK language

Reviewed By: patapizza

Differential Revision: D13024050

fbshipit-source-id: e5be933fff8041c42d8fd26ddaf919584912a012
2018-11-19 04:31:03 -08:00
Vitaly Shaydurov
3c5790f605 TimeGrain,Duration/UK: added dimensions
Summary:
- added TimeGrain dimension for UK language
- added Duration dimension for UK language

Reviewed By: patapizza

Differential Revision: D12871569

fbshipit-source-id: 356c2031aa9582620be11ea634c854c0d96ebbeb
2018-11-19 04:31:03 -08:00
Giovanni Cappellotto
39a6a4c6ec Add missing pattern match
Summary:
`fromName` was missing a pattern to match the new `"credit-card-number"` dimension string representation.

Adding also `CreditCardNumber.Tests.tests` to the `Dimensions Tests` test group.

Reviewed By: chinmay87

Differential Revision: D13072125

fbshipit-source-id: 792727ef038d26ae12ceea909fa726692d56db6d
2018-11-15 07:16:57 -08:00
Giovanni Cappellotto
1052914cb4 Credit cards
Summary:
Add the ability to parse credit cards and identify the issuer in Duckling.

Credit card numbers are a sequence of 8 to 19 digits, with a few specific digits at the start that identify the card issuer. They also satisfy Luhn checksum (https://en.wikipedia.org/wiki/Luhn_algorithm).

Reviewed By: chinmay87

Differential Revision: D13014623

fbshipit-source-id: 96586e074777ae90a4a39c515648c31a8111f0c2
2018-11-14 11:16:03 -08:00
Julien Odent
d0516324ec Time/VI: don't parse ngày
Summary:
Fixes https://github.com/facebook/duckling/issues/273.
`ngày` can mean day or date in Vietnamese.
Fixed bad Christmas regex.

Reviewed By: chinmay87

Differential Revision: D12955229

fbshipit-source-id: 071b623e581d36f7db4bf356219647147a087a25
2018-11-07 09:45:53 -08:00
Kumar Shridhar
975489fed6 Added German+Icelandic spelled out Email
Summary: Pull Request resolved: https://github.com/facebook/duckling/pull/272

Reviewed By: chinmay87

Differential Revision: D12955290

Pulled By: patapizza

fbshipit-source-id: 671ee50c92c031762aa147dc6bd98f374693ba6b
2018-11-07 09:15:53 -08:00
PhalPheaktra Chhaya
3374f4ea50 Add Volume/KM and Distance/KM (#256)
Summary:
Hello.
I have updated Quantity/KM and added 3 more dimensions (Volume/KM, Time/KM, and Distance/KM) for KM.
Please take a look at it.
Pull Request resolved: https://github.com/facebook/duckling/pull/256

Reviewed By: patapizza

Differential Revision: D9758093

Pulled By: xhavokx

fbshipit-source-id: 5a800fd443789795b8e63c88dade3785cfca706c
2018-10-31 16:45:58 -07:00
Akhil Ravidas
e349942541 Support malayalam ordinals
Summary: Add support for malayalam ordinals.

Reviewed By: patapizza

Differential Revision: D10097075

fbshipit-source-id: 90be5f05d1f9cf4cc6558a4583c8c72518fe0ab0
2018-09-28 10:30:54 -07:00
Akhil Ravidas
c46dbb0d92 Support malayalam numerals
Summary: Add support for malayalam numerals in duckling.

Reviewed By: patapizza

Differential Revision: D10075358

fbshipit-source-id: ce7fee2a71d862391980481b018c513981249f3d
2018-09-28 10:30:54 -07:00
pheaktra21
cd33b26acf Add support for Ordinal, Quantity, & Temperature dimension for KM. (#251)
Summary:
Hello.
I added 3 new dimensions: Ordinal, Quantity, & Temperature. And I also added more data to Numeral dimension.
Thank you!
Pull Request resolved: https://github.com/facebook/duckling/pull/251

Reviewed By: beauby

Differential Revision: D9656646

Pulled By: patapizza

fbshipit-source-id: f17797be811d58b0b5bcd02b83c0a699650c9f6e
2018-09-06 07:01:15 -07:00
kumar-shridhar
4a77765134 Icelandic support
Summary: Pull Request resolved: https://github.com/facebook/duckling/pull/247

Reviewed By: beauby

Differential Revision: D9656660

Pulled By: patapizza

fbshipit-source-id: 993baa9a0e442fc6f6f813a3af717b700c4d89e6
2018-09-06 06:46:02 -07:00
David Magaltadze
ab3b789896 KA AmountOfMoney (#245)
Summary:
Added support for AmountOfMoney for KA
Pull Request resolved: https://github.com/facebook/duckling/pull/245

Differential Revision: D9510854

Pulled By: patapizza

fbshipit-source-id: 966c0f69193dd37ed458339fbc381ca640a3a179
2018-08-28 17:00:50 -07:00
Nathan Hu
eb0b84a5aa Time/EN: add <datetime> - <datetime> (interval) timezone rule
Summary: Introducing `hasTimezone` flag to prevent double timezone parsing.

Reviewed By: patapizza

Differential Revision: D9317892

fbshipit-source-id: 0e9246abeb65c5fa61725f9b78a1421d85263bdc
2018-08-24 11:45:45 -07:00
David Magaltadze
010146d2f4 GE + Duration, Ordinal, TimeGrain, Time (#221)
Summary:
Added initial support for ka_GE for

- Duration
- Ordinal
- TimeGrain
- Time
Pull Request resolved: https://github.com/facebook/duckling/pull/221

Differential Revision: D8840919

Pulled By: haoxuany

fbshipit-source-id: 80f94fbaf25786b0f58cd6598c29cd3663858809
2018-08-22 18:30:48 -07:00
Gerben Janssen van Doorn
f70f991b38 Added support Lao numerals
Summary: Duckling didn't support Lao numerals, this diff adds it.

Reviewed By: patapizza

Differential Revision: D9323242

fbshipit-source-id: 5dad0d4dfb4843281a327947690e664c91ab8f1a
2018-08-17 10:31:17 -07:00
Julien Odent
e818b4ce9e Time/EN: Restrict "on" absorption to days
Summary:
We were absorbing "on" followed by any Time token.
Restricting to days of weeks and days of months only.

https://github.com/facebook/duckling/issues/217

Reviewed By: chinmay87

Differential Revision: D9313306

fbshipit-source-id: a0c33658855af8f69631d4d1879cc27062c26a69
2018-08-15 01:16:12 -07:00
PhalPheaktra Chhaya
b541354c31 Add Numeral dimension for new language KM. (#234)
Summary:
Hello,
I am new to Haskell, but I would like to add Khmer language (KM) to Duckling.
I have tried to extended Duckling by adding Numeral dimension for new language KM.
Please have a look at it and see what we can improve.

Thanks!
Pull Request resolved: https://github.com/facebook/duckling/pull/234

Reviewed By: blandinw

Differential Revision: D9032639

Pulled By: chinmay87

fbshipit-source-id: 7db19edf732fe6500629cc89e18e0655d7bbc48b
2018-08-03 14:45:55 -07:00
Nivin Lawrence
e279c5642e added tamil ordinal
Summary: support tamil ordinal

Reviewed By: girifb

Differential Revision: D8713963

fbshipit-source-id: c0760f0cf13d983473d29e18425ba926584c3072
2018-07-16 17:15:53 -07:00
Arunavha Chanda
d5555d0149 Numeral/BN: Adding Bengali numeral support to Duckling
Summary: Added support for Bengali numerals

Reviewed By: patapizza

Differential Revision: D8730468

fbshipit-source-id: dc36017e24d796f35abc477a0b8b317218c64a6a
2018-07-09 12:30:30 -07:00
Tero Laxström
4ed1ed83ed Basics for Finnish (#210)
Summary:
Adds Locale and Numeral for Finnish
Closes https://github.com/facebook/duckling/pull/210

Reviewed By: JonCoens

Differential Revision: D8430386

Pulled By: patapizza

fbshipit-source-id: a3c8b3b3419b7f43e2ef332cdb1fb8fc07da3bec
2018-06-19 10:45:27 -07:00
Anshuman Chhabra
5ab49060e1 Added Temperature support for Hindi (HI) language (#206)
Summary:
This PR adds Temperature dimension support for Hindi (HI). Let me know if there are any changes to be made :)

Cheers!
Closes https://github.com/facebook/duckling/pull/206

Reviewed By: haoxuany

Differential Revision: D8381472

Pulled By: patapizza

fbshipit-source-id: a085ba73910a80a51385f01475d9ba75f72fd4c5
2018-06-13 17:00:35 -07:00
Julien Odent
e8286f762c Numeral/RO: Fix multipliers with values above 20
Summary: In Romanian, for numerals above 20, we say "20 de milioane", not "20 milioane".

Reviewed By: haoxuany

Differential Revision: D8334109

fbshipit-source-id: a7fc83440334ab9b1f0511f315029e28449f9771
2018-06-11 11:00:38 -07:00
Julien Odent
483df26347 AmountOfMoney/RO: Fix for values above 20
Summary: In Romanian, we say "19 dolari", but for numerals above 20, it's "20 de dolari" (not "20 dolari").

Reviewed By: haoxuany

Differential Revision: D8324582

fbshipit-source-id: ffd29fa11bbc3a3f0da314f327d26c7b7890545f
2018-06-07 16:16:45 -07:00
Julien Odent
f272a175d4 Time/NL: Fix Sinterklaas for Belgium
Summary: In south Netherlands and in Belgium, Sinterklaas is celebrated on December 6.

Reviewed By: mullender

Differential Revision: D8281767

fbshipit-source-id: 4fc71783b9f543c4dfb04b8ba8481935cfb30dd8
2018-06-05 16:00:43 -07:00
Aaron Yue
b64ff2f21b Time/EN: add <time> for <duration> rules
Summary: add rule for "from <time> for <duration>" and "<time> for <duration>" (symmetric to "for <duration> from <time>")

Reviewed By: patapizza

Differential Revision: D8195733

fbshipit-source-id: 86a0a24d847ea004aeed89ce8a144cb3311aee59
2018-05-31 09:30:30 -07:00
Edward Akerboom
ddbb6fdff8 Remove am/pm rule
Summary:
See https://github.com/facebook/duckling/issues/196
Closes https://github.com/facebook/duckling/pull/199

Reviewed By: patapizza

Differential Revision: D8098984

Pulled By: xhavokx

fbshipit-source-id: e6c7f94b8cda09c5f96c75052d333bced0c85ac2
2018-05-23 06:15:35 -07:00
RIAN DOUGLAS
ecceacb820 Implement Temperature intervals
Summary: Make some changes to allow the TemperatureValue type to support intervals.

Reviewed By: patapizza

Differential Revision: D7680835

fbshipit-source-id: d18c67fbb03f14d61cddb53dae3c544513cc67f2
2018-05-21 12:00:41 -07:00
RIAN DOUGLAS
884904b5ca Implement handling of "grand" for EN_?? locales
Summary: Add implementation of "a grand" and "<num> grand" for AU, BZ, CA, GB, IE, IN, JM, NZ, PH, TT, ZA locales. Some resolve to the local currency (AU, IN), others resolve to Dollar (NZ, PH).

Reviewed By: patapizza

Differential Revision: D7943186

fbshipit-source-id: c71ab462fa9df0ee65223ee82dc2c98457a4e13b
2018-05-17 15:45:27 -07:00
Anshuman Chhabra
8237ef4503 Added Duration support to Hindi (HI)
Summary:
Hello!

I have added __Hindi (HI)__ support for the __Duration__ dimension in ```Duckling/Duration/Rules.hs```. Written tests as well in ```Duckling/Duration/Corpus.hs``` and everything is passing and verified (`253 tests passed`).

Cheers!
Closes https://github.com/facebook/duckling/pull/194

Reviewed By: patapizza

Differential Revision: D7999571

Pulled By: chinmay87

fbshipit-source-id: deb5d60ba7f7ecbc2aa4d97ce5fb96e9bbe63b3d
2018-05-17 14:46:21 -07:00
RIAN DOUGLAS
963aa6b465 Implement handling of "grand" for Ca and GB
Summary: Add implementation of "a grand" and "<num> grand" for EN_CA and EN_GB locales

Reviewed By: patapizza

Differential Revision: D7916643

fbshipit-source-id: 0cd55f17ec522c0334f48436a8a8cc19e0560b0b
2018-05-09 14:46:16 -07:00
Ziyang Liu
c84166d708 Add Quantity/ZH
Summary:
Fixes #181.
Closes https://github.com/facebook/duckling/pull/182

Reviewed By: patapizza

Differential Revision: D7891288

Pulled By: haoxuany

fbshipit-source-id: faa505dbf149c728545a92579cb2e92a89cd5cb8
2018-05-07 09:45:32 -07:00
RIAN DOUGLAS
f328821bbc Add support for using 'grand' to refer to 1,000's of currency in EN
Summary: Added support for ",number> grand" and "a grand" to result in thousands of currency amount

Reviewed By: haoxuany

Differential Revision: D7688809

fbshipit-source-id: 72a81c7a1c48329f85c9f525b72b00c479a9edb0
2018-05-03 14:45:25 -07:00
Ziyang Liu
a3b35880e5 Change value in Entity to typed value instead of JSON
Summary:
Modified `Entity` to use the new `ResolvedVal` data type. Other changes follow naturally. Related issues: https://github.com/facebook/duckling/issues/121 and https://github.com/facebook/duckling/issues/172

Now one can pattern match on the output value, for instance:

```
{-# LANGUAGE GADTs #-}

import Data.Text
import Duckling.Core
import Duckling.Testing.Types
import qualified Duckling.PhoneNumber.Types as PN

parsePhoneNumber :: Text -> Text
parsePhoneNumber input =
  case value entity of
    (RVal PhoneNumber (PN.PhoneNumberValue v)) -> v
    where
    (entity:_) = parse input testContext testOptions [This PhoneNumber]
```

Reviewed By: patapizza

Differential Revision: D7502020

fbshipit-source-id: 76ba7b315cfd0d2c61ff95c855b7c95efc0a401c
2018-04-20 14:18:47 -07:00
Aaron Yue
4df76289bc Implement AmountOfMoney
Summary: implement AmountOfMoney rules and corpus for ZH

Reviewed By: patapizza

Differential Revision: D7508507

fbshipit-source-id: 3591b399a9880c5278587979c6576720343cc123
2018-04-17 16:45:32 -07:00
Aaron Yue
4ef255e577 Generalize and expand digit specifier usage for hanzi
Summary:
generalize chinese digit specifier (十百千万亿) parsing, and add hanzi tests
These digits specifiers can be parsed as (<num><speci>)<num>,
by using the multiplicater value <num><speci>, and a connect function that adds them together
(two cases, skipping digits [which requires a 零 in between], and digits in consecutive locations).
Note that 个 is technically a digit specifier,
but in Chinese it is never used directly as a numeral specifier, and always as a counter.

Reviewed By: zliu41

Differential Revision: D7424249

fbshipit-source-id: 20a85a7df1f908ee9879e92b904178fa26a9a5e5
2018-04-12 10:00:33 -07:00
Souvik Ghosh
ced73dcdcb AmountOfMoney NL Support
Summary:
Hello,
I have added new directory for AmountOfMoney NL Support
Added two files
- Duckling/AmountOfMoney/NL/Corpus.hs
- Duckling/AmountOfMoney/NL/Rules.hs

Updated File
- Duckling/Rules/NL.hs

Added test cases
- tests/Duckling/AmountOfMoney/NL/Tests.hs

Updated Test file
- tests/Duckling/AmountOfMoney/Tests.hs

Updated
duckling.cabal

Thanks for the review and the latest merge. Looking forward

Regards
Closes https://github.com/facebook/duckling/pull/173

Reviewed By: JonCoens

Differential Revision: D7592192

Pulled By: patapizza

fbshipit-source-id: 5895c29bf7f1033e4ffd791d5915a16d230e9375
2018-04-11 17:15:30 -07:00
Julien Odent
120f569ed0 Time/EN: Fix "in" + year
Summary: Now that years are latent, let's absorb the "in" and make them not latent.

Reviewed By: zliu41

Differential Revision: D7587599

fbshipit-source-id: 61a19ac389244df491591d78c28f0301f9124439
2018-04-11 11:00:30 -07:00
Aaron Yue
878bcb9277 Show input for ambiguous parse failures
Summary: show the input that an ambiguous parse is failing at

Reviewed By: patapizza

Differential Revision: D7502191

fbshipit-source-id: 9f0fbf8301413d9007236ba5b6af1f4b41c20269
2018-04-04 10:30:32 -07:00
Aaron Yue
3629fbd503 Prompt ambiguous parses in corpus tests
Summary:
During ranking, due to how candidates are ordered, it is completely possible to have multiple correct candidates
have the exact same rank (equal range and exact equal score). In this case `analyze` returns all of them, which gets
misinterpreted as having multiple tokens in output rather than multiple solutions. Checks this case and gives the
correct prompt for ambiguous parses.

Reviewed By: patapizza

Differential Revision: D7489391

fbshipit-source-id: b66947e37bddb3ac6273843dd79b559aff9d0083
2018-04-03 17:00:27 -07:00
Ziyang Liu
22d11c055b Add ZH distance support
Summary: Add ZH distance support

Reviewed By: patapizza

Differential Revision: D7341977

fbshipit-source-id: 301d90515c492da9fa4c4faf5bcc3351eacbed1b
2018-03-21 12:15:28 -07:00
hend
f70b4ebe07 add distance to SV
Summary: Closes https://github.com/facebook/duckling/pull/168

Reviewed By: chinmay87

Differential Revision: D7322559

Pulled By: patapizza

fbshipit-source-id: 3d4b86068c05a7a82442bc7c182b80b5cf75ec90
2018-03-21 11:15:25 -07:00
yasen-yankov
21c3b32e4d Added TimeGrain, Duration, and Ordinal
Summary: Closes https://github.com/facebook/duckling/pull/164

Reviewed By: chinmay87

Differential Revision: D7280110

Pulled By: patapizza

fbshipit-source-id: d98ddd900fe83f06b28afd39ea3311f42716288c
2018-03-19 18:00:29 -07:00
Giri Anantharaman
519c9519a3 Support Tamil numerals
Summary:
* Setup Tamil (TA) language
* Added Numeral Dimension

Reviewed By: patapizza

Differential Revision: D7323636

fbshipit-source-id: 4b1a42197ff4799880cded9ce86b8d7fae1507bc
2018-03-19 16:45:36 -07:00
Chinmay Deshmukh
5ac990bbe2 Return latent entities
Summary: Add an option to return latent time entities. This can be used when one is pretty certain that the input contains a datetime.

Reviewed By: patapizza

Differential Revision: D7254245

fbshipit-source-id: e9e0503cace2691804056fcebdc18fd9090fb181
2018-03-19 14:45:27 -07:00
yasen-yankov
1493d44465 Added BG distance
Summary: Closes https://github.com/facebook/duckling/pull/162

Reviewed By: chinmay87

Differential Revision: D7239243

Pulled By: patapizza

fbshipit-source-id: a5518219a67fa46bb06e97eee3dfd07ab683162f
2018-03-13 15:45:32 -07:00
Ezgi Çiçek
2254034e62 Add Node to Entity to pass along the parse tree information
Summary:
- add `Node` field to `Entity`
- ignore `Node` field of `Entity` for toJSON for now (will be fixed later)
- change `Debug.hs` so that we print the respective Entity's Node
- add wildcard for the new Node field in `parseTest` of Api/Tests.hs`

Reviewed By: patapizza

Differential Revision: D7174696

fbshipit-source-id: 240e4c53b72323b500ac58a74f873ce247bb3387
2018-03-07 10:45:28 -08:00