Commit Graph

91 Commits

Author SHA1 Message Date
Jens Persson
41f140992d Add support for larger spelled-out Danish ordinals (#346)
Summary:
Adds support for larger spelled-out Danish ordinal number expressions, like
treoghalvfemsindstyvende (93rd)
or
tohundrede og femogfyrrende (245th)
Pull Request resolved: https://github.com/facebook/duckling/pull/346

Reviewed By: patapizza

Differential Revision: D14476918

Pulled By: chinmay87

fbshipit-source-id: eb20ee8d304f291ff4ab2b28c4e272a9d447396e
2019-05-23 16:02:56 -07:00
Julien Odent
bf89e34365 Relicense to BSD3
Reviewed By: JoelMarcey

Differential Revision: D15439223

fbshipit-source-id: c5af3cb06318748142fe503945b38beffadfc28a
2019-05-22 10:46:39 -07:00
Nathan Hausman
cbfca8ccc3 Support Arabic decimals and comma separated integers
Summary: Adding support for decimal numbers written in Arabic script.

Reviewed By: patapizza

Differential Revision: D14700060

fbshipit-source-id: 3261c36dde17dc9e589eac6b6521da6dece814f2
2019-04-03 13:07:23 -07:00
Nathan Hausman
f6399c8462 Support Arabic numeric integer and fractions
Summary: Adding support for Arabic numbers written in Arabic script.

Reviewed By: patapizza

Differential Revision: D14697405

fbshipit-source-id: 1162319afda55bdfdcb132932fa7e3a99fdcda15
2019-04-03 08:33:26 -07:00
Nathan Hausman
b90ff9a64e Support Arabic phone numbers
Summary: Support Arabic phone numbers for AR locales.

Reviewed By: patapizza

Differential Revision: D14591780

fbshipit-source-id: 93875c5b407c3fe2a06276c6fa3af2a26c91639e
2019-03-29 09:48:20 -07:00
ilakli
25ff06c165 KA - Improvements in Time, Numeral, Duration (#299)
Summary:
- Improved understanding of past periods and dates which include days, months and years.
- Added support for quarters.
- Fixed unnecessary two or more results on some queries.
Pull Request resolved: https://github.com/facebook/duckling/pull/299

Differential Revision: D13878236

Pulled By: patapizza

fbshipit-source-id: 5ae4ac96dd5d75f54ed1567017a81e5a9c8bcc83
2019-02-22 18:16:32 -08:00
Jens Persson
e176cd0195 fix veintodós spelling (#325)
Summary:
This seems to have existed earlier but have been removed. The correct spelling seems to be with the accent though.
Pull Request resolved: https://github.com/facebook/duckling/pull/325

Differential Revision: D14059697

Pulled By: patapizza

fbshipit-source-id: 7c2aafc555a31bff8962032613b8a1da0edc8dc9
2019-02-12 17:31:21 -08:00
akki2825
a2494caf61 add Kannada support (for Numeral)
Summary: Pull Request resolved: https://github.com/facebook/duckling/pull/322

Reviewed By: patapizza

Differential Revision: D13888983

Pulled By: haoxuany

fbshipit-source-id: b7a3b5c72dae0ccc637cf6faa14670ebf868bb50
2019-02-08 11:31:39 -08:00
Micha Molko
301f74ea2e AmountOfMoney/HE Extend support
Summary: Improve support for amount of money in Hebrew

Reviewed By: patapizza

Differential Revision: D13685915

fbshipit-source-id: f9796f52b4d011bdc5e1ef903d5e3dc789bfcd04
2019-02-06 02:32:24 -08:00
Hamza Kwisaba
392e584d1b swahili numerical support
Summary: Adding support for swahili numerals

Reviewed By: patapizza

Differential Revision: D13637092

fbshipit-source-id: 6fe0facfa74caae6fd00e0e84b09571aca616f21
2019-01-18 01:16:13 -08:00
Michele Riva
9a45a0cf7b Added IT Rules and Corpus for AmountOfMoney (#284)
Summary:
Hi,
I've added Rules and Corpus for Italian language on "AmountOfMoney".
Running `stack test`, I always get `empty result on "10 000 dollari"`.
I've checked my code more than once but I can't figure out what is not working. I made a PR in order to make it easier for you to understand what's going wrong :)

I really hope that my contribution can help.
Regards!
Pull Request resolved: https://github.com/facebook/duckling/pull/284

Reviewed By: chinmay87

Differential Revision: D13551890

Pulled By: patapizza

fbshipit-source-id: c1b448e44c2c6720ad93f61c6396f4d75231131e
2019-01-02 14:01:53 -08:00
uugan
69ffb0f39d Mongolian (#277)
Summary:
MN Setup+Numeral
Pull Request resolved: https://github.com/facebook/duckling/pull/277

Reviewed By: chinmay87

Differential Revision: D13028169

Pulled By: patapizza

fbshipit-source-id: 7d587300918b3ed6211b6ce4147350147b4dc89e
2018-11-26 15:31:08 -08:00
Chinmay Deshmukh
9077f9e147 AmountOfMoney,Numeral/EN: Add support for lakh and crore
Summary:
Add support for lakh (1e5) and crore (1e7) in Numeral.

This will help resolve "9-7 lakh rupees" to (Rs. 9, Rs. 700000). Next step is to resolve it to (Rs. 900000, Rs. 700000)

Reviewed By: patapizza

Differential Revision: D10417678

fbshipit-source-id: 670115f22015698ff8c7a9d96afd624efd4c864e
2018-10-17 10:30:50 -07:00
Akhil Ravidas
c46dbb0d92 Support malayalam numerals
Summary: Add support for malayalam numerals in duckling.

Reviewed By: patapizza

Differential Revision: D10075358

fbshipit-source-id: ce7fee2a71d862391980481b018c513981249f3d
2018-09-28 10:30:54 -07:00
Konstantin Kloster
ee24a4196d small bugfix in german numeral dimension (#257)
Summary:
There is a small bug in the German numeral dimension. The number "fünfzehn"(15) cannot be recognized correctly due to a typo. This pull request should fix it:)
Pull Request resolved: https://github.com/facebook/duckling/pull/257

Reviewed By: girifb

Differential Revision: D9810012

Pulled By: patapizza

fbshipit-source-id: e2408a2dc237590b24840260e2a97e9b5c8231f3
2018-09-13 14:30:44 -07:00
pheaktra21
cd33b26acf Add support for Ordinal, Quantity, & Temperature dimension for KM. (#251)
Summary:
Hello.
I added 3 new dimensions: Ordinal, Quantity, & Temperature. And I also added more data to Numeral dimension.
Thank you!
Pull Request resolved: https://github.com/facebook/duckling/pull/251

Reviewed By: beauby

Differential Revision: D9656646

Pulled By: patapizza

fbshipit-source-id: f17797be811d58b0b5bcd02b83c0a699650c9f6e
2018-09-06 07:01:15 -07:00
kumar-shridhar
4a77765134 Icelandic support
Summary: Pull Request resolved: https://github.com/facebook/duckling/pull/247

Reviewed By: beauby

Differential Revision: D9656660

Pulled By: patapizza

fbshipit-source-id: 993baa9a0e442fc6f6f813a3af717b700c4d89e6
2018-09-06 06:46:02 -07:00
jfulse
b9d39f723e Norwegian numeral improvements (#243)
Summary:
Some small improvements to Norwegian numerals:

- Parse powers of ten with spaces as well as dots: `10 000 == 10.000`
- Some more textual powers of ten: `milliard`, `billion`, ...
- Textual numbers from 21 to 99: `tjueen`, `tjueto`, ...
   (they were already parsed with spaces, as in `tjue en`, but not without)
Pull Request resolved: https://github.com/facebook/duckling/pull/243

Reviewed By: patapizza

Differential Revision: D9505835

Pulled By: haoxuany

fbshipit-source-id: e42a290b3dd2d5e6a82f00514617b4c3c4dd3416
2018-08-31 17:46:08 -07:00
Gerben Janssen van Doorn
f70f991b38 Added support Lao numerals
Summary: Duckling didn't support Lao numerals, this diff adds it.

Reviewed By: patapizza

Differential Revision: D9323242

fbshipit-source-id: 5dad0d4dfb4843281a327947690e664c91ab8f1a
2018-08-17 10:31:17 -07:00
PhalPheaktra Chhaya
b541354c31 Add Numeral dimension for new language KM. (#234)
Summary:
Hello,
I am new to Haskell, but I would like to add Khmer language (KM) to Duckling.
I have tried to extended Duckling by adding Numeral dimension for new language KM.
Please have a look at it and see what we can improve.

Thanks!
Pull Request resolved: https://github.com/facebook/duckling/pull/234

Reviewed By: blandinw

Differential Revision: D9032639

Pulled By: chinmay87

fbshipit-source-id: 7db19edf732fe6500629cc89e18e0655d7bbc48b
2018-08-03 14:45:55 -07:00
jfulse
e1088c1856 Norwegian improvements (#232)
Summary:
Some small improvements to the Norwegian implementation:

- The written number 8 had a typo: "otte" -> "åtte"
- Add support for half an hour before as e.g. "halv to"
- Add support for alternative clock denotation "klokka"
- Add support for alternative tomorrow denotation "i morra"
Pull Request resolved: https://github.com/facebook/duckling/pull/232

Reviewed By: girifb

Differential Revision: D8986441

Pulled By: chinmay87

fbshipit-source-id: 286617d30415febe1f0eda4bc7475ca5c9610734
2018-07-25 13:15:50 -07:00
Julien Odent
9c367ab6cd Don't accept dashes (-) as token separators
Summary:
This is causing some issues, e.g. `20-30` resolving to 8:30pm (latent).
Updating `Numeral` rules to account for that (`EN`, `FR`, `NL` following tests).

Differential Revision: D8854891

fbshipit-source-id: ba17099b014d9cf2f48a7d85147cc890b02578f5
2018-07-16 06:00:40 -07:00
Arunavha Chanda
d5555d0149 Numeral/BN: Adding Bengali numeral support to Duckling
Summary: Added support for Bengali numerals

Reviewed By: patapizza

Differential Revision: D8730468

fbshipit-source-id: dc36017e24d796f35abc477a0b8b317218c64a6a
2018-07-09 12:30:30 -07:00
Cuong Dinh Tri Nguyen
4e11613d39 Supported AmountOfMoney intervals and improved Numeral
Summary:
Support amount of money intervals for VI

Modify ruleNg, ruleDollar, and ruleVND to better capture the usage of VI

In Numeral, add "ngàn" - a common synonym of "nghìn", and "chục" - colloquially used to count tens. Remove "?" in regex of other words as it does not make sense.

Reviewed By: haoxuany

Differential Revision: D8734066

fbshipit-source-id: 15f879ab796025882c85f0ce9f1677c501b364a0
2018-07-09 10:00:42 -07:00
Tero Laxström
4ed1ed83ed Basics for Finnish (#210)
Summary:
Adds Locale and Numeral for Finnish
Closes https://github.com/facebook/duckling/pull/210

Reviewed By: JonCoens

Differential Revision: D8430386

Pulled By: patapizza

fbshipit-source-id: a3c8b3b3419b7f43e2ef332cdb1fb8fc07da3bec
2018-06-19 10:45:27 -07:00
David Magaltadze
713e5db9d6 Added support of Numeral for ka_GE (#211)
Summary:
* `ka_GE` locale
* full support for Numeral
Closes https://github.com/facebook/duckling/pull/211

Reviewed By: chinmay87

Differential Revision: D8456241

Pulled By: patapizza

fbshipit-source-id: 35fb432191bd20a4965503efde94c5ea29d0c50d
2018-06-18 13:31:14 -07:00
Julien Odent
e8286f762c Numeral/RO: Fix multipliers with values above 20
Summary: In Romanian, for numerals above 20, we say "20 de milioane", not "20 milioane".

Reviewed By: haoxuany

Differential Revision: D8334109

fbshipit-source-id: a7fc83440334ab9b1f0511f315029e28449f9771
2018-06-11 11:00:38 -07:00
Andreea Danielescu
bee838e1c5 AmountOfMoney/RO Support intervals
Summary: Added support for intervals (between X and Y), min, max and approximates for currency in RO.

Reviewed By: patapizza

Differential Revision: D8322948

fbshipit-source-id: 462cfd5575c87e757d2c35c8078f539af3e8150f
2018-06-08 09:00:31 -07:00
Julien Odent
567450beea Distance/RO: Fix for values above 20, refactoring
Summary:
* One rule to rule them all, to easily duplicate with "de" suffix.
* This would actually allow things like "4 de metri", though it's fine as it doesn't alter meaning.

Reviewed By: haoxuany

Differential Revision: D8325328

fbshipit-source-id: 3ec0f30431f3cb00152cc9509f6052b1ae29cd08
2018-06-07 16:45:43 -07:00
Abdallatif
a00a0d7bdf fix dual big numbers in Arabic
Summary:
for 200, 2000, 2000000
Closes https://github.com/facebook/duckling/pull/202

Reviewed By: adelnobel

Differential Revision: D8257813

Pulled By: patapizza

fbshipit-source-id: d83ea31b9fdf4d28a61b75e84583ef0d7e7bea30
2018-06-04 14:45:37 -07:00
Abdallatif
9f2fffec23 fix million in ar
Summary:
Fixes https://github.com/wit-ai/wit/issues/1080

Closes https://github.com/facebook/duckling/pull/195

Reviewed By: adelnobel

Differential Revision: D8030001

Pulled By: patapizza

fbshipit-source-id: 0dc0253ff8527c94c8e81a891f33898764336d6d
2018-05-16 14:45:42 -07:00
Ziyang Liu
5460d8df0e Support custom dimensions
Summary:
Support custom dimensions

Had to move the definition of `Dimension` from `Duckling.Dimensions.Types` to `Duckling.Types` to avoid cyclic imports between these two modules.

A sample custom dimension is in `exe/CustomDimensionExample.hs`.

Limitations of custom dimensions:

- All rules for a custom dimension must be in the same module with the definition of the custom dimension. Otherwise there will be cyclic imports, because the definition of the dimension and the rules refer to each other.
- The custom dimension must be specified when using `parse`, since there's no way to get all the existing custom dimensions.

Reviewed By: patapizza

Differential Revision: D7630360

fbshipit-source-id: 30e12dcb33611f5692c4f5949de377bf61b75e1e
2018-04-19 15:30:51 -07:00
Quan
c166179d89 Use comma "," as decimal separator for VI
Summary:
In Vietnam, we use comma "," as decimal separator instead of dot "."
Closes https://github.com/facebook/duckling/pull/176

Reviewed By: patapizza

Differential Revision: D7601328

Pulled By: panagosg7

fbshipit-source-id: 7f5725006f9054fe45a24757ec2abc36b7aa6605
2018-04-18 23:15:30 -07:00
Aaron Yue
babe317723 fix intersect rule to work with negative numbers
Summary:
Numerals that require intersection with negative numbers don't work, since the negative sign gets parsed
before the intersect rules happen. This fixes it by adding a guard in for positive in the intersection rule.

Reviewed By: patapizza

Differential Revision: D7592225

fbshipit-source-id: 2bc9c708cadeea4012c1f3ef487c61a144325f2a
2018-04-13 19:45:40 -07:00
Aaron Yue
fe0807dced Use lambda-case for Rule production
Summary: change Rule production functions to lambda-case

Reviewed By: patapizza

Differential Revision: D7424413

fbshipit-source-id: edac0290d310578f633ff0208434d1eca038ad9c
2018-04-12 10:15:57 -07:00
Aaron Yue
4ef255e577 Generalize and expand digit specifier usage for hanzi
Summary:
generalize chinese digit specifier (十百千万亿) parsing, and add hanzi tests
These digits specifiers can be parsed as (<num><speci>)<num>,
by using the multiplicater value <num><speci>, and a connect function that adds them together
(two cases, skipping digits [which requires a 零 in between], and digits in consecutive locations).
Note that 个 is technically a digit specifier,
but in Chinese it is never used directly as a numeral specifier, and always as a counter.

Reviewed By: zliu41

Differential Revision: D7424249

fbshipit-source-id: 20a85a7df1f908ee9879e92b904178fa26a9a5e5
2018-04-12 10:00:33 -07:00
Giri Anantharaman
519c9519a3 Support Tamil numerals
Summary:
* Setup Tamil (TA) language
* Added Numeral Dimension

Reviewed By: patapizza

Differential Revision: D7323636

fbshipit-source-id: 4b1a42197ff4799880cded9ce86b8d7fae1507bc
2018-03-19 16:45:36 -07:00
Chinmay Deshmukh
5ac990bbe2 Return latent entities
Summary: Add an option to return latent time entities. This can be used when one is pretty certain that the input contains a datetime.

Reviewed By: patapizza

Differential Revision: D7254245

fbshipit-source-id: e9e0503cace2691804056fcebdc18fd9090fb181
2018-03-19 14:45:27 -07:00
Panagiotis Vekris
169df6b46b ruleSkipHundreds only accepts alphabetic numerals
Summary:
Before, duckling would parse `"1 2 2"` as the three digit number 122 through
`ruleSkipHundreds`. This, however, allowed the string `"Pay Kiran1 10eur"` to be
parsed as "110 EUR", which was reported in https://github.com/facebook/duckling/issues/159.

This change only accepts alphabetic numerals to pass through `ruleSkipHundreds`
(for example `"one twenty two"`), which presumably was the original intention of
this rule. This fixes the above issue, without any change in Corpus.

Reviewed By: patapizza

Differential Revision: D7151934

fbshipit-source-id: 024a7a0b6a53beb3a0d42d4bb7f542ce3b05726b
2018-03-06 17:00:29 -08:00
Julien Odent
aed5b8f779 Numeral: don't compose negative numbers, fix double negatives
Summary:
* added `isMultipliable` helper and used that in patterns along with `isPositive`
* fixed double negatives in most languages

Reviewed By: niteria

Differential Revision: D7034982

fbshipit-source-id: a0bb67056d3107167830ece0c34d761c5563c5a7
2018-02-21 12:02:45 -08:00
bidhan-a
43079e7113 Setup Nepali (NE) and add Numeral dimension
Summary:
- Setup Nepali (NE) language
- Add basic Numeral dimension
Closes https://github.com/facebook/duckling/pull/156

Reviewed By: JonCoens

Differential Revision: D6965558

Pulled By: patapizza

fbshipit-source-id: f46c9b104d4345f20bd0cf53f8c9c8754855f314
2018-02-13 07:45:31 -08:00
Julien Odent
b00fa512af AmountOfMoney: fixes
Summary:
* don't recursively compose cents
* don't allow decreasing ranges

Reviewed By: blandinw

Differential Revision: D6849132

fbshipit-source-id: ed6ca30388642c21e677a628971747a4fb3dfbef
2018-01-30 14:45:36 -08:00
Julien Odent
bef7a44fa8 Remove redundant brackets and language pragmas
Summary: .

Reviewed By: JonCoens

Differential Revision: D6838082

fbshipit-source-id: 94757bdb80c6d3c29a7a6554429940a1b7403108
2018-01-29 16:45:28 -08:00
Abdallatif Sulaiman
ce2f48fbb5 Quickfix for Arabic Numeral
Summary: Closes https://github.com/facebook/duckling/pull/137

Reviewed By: adelnobel

Differential Revision: D6660959

Pulled By: patapizza

fbshipit-source-id: 51169c63558b5a35f16bdaaada9649c88a4d2994
2018-01-05 11:45:40 -08:00
Abdallatif Sulaiman
1393098bcc Added Time Dimension to Arabic
Summary:
Hi, in this pr:
* Added time dimension to Arabic language, thanks to Hussein-Dahir & Yazeed-Obaid for writing time corpus.
* Fixed some bugs in numeral & ordinals and added more test cases for them.

Also, I don't really understand why do we use classifiers in time dimension?
Closes https://github.com/facebook/duckling/pull/123

Reviewed By: blandinw

Differential Revision: D6583313

Pulled By: patapizza

fbshipit-source-id: f7acdef0c032d7b7fd7d224832fdaf484d2df825
2017-12-19 14:30:42 -08:00
Newinfinite007
c133bad24a Hindi Language Numeral Dimension(minimalistic model). Tests passed.
Summary: Closes https://github.com/facebook/duckling/pull/119

Reviewed By: JonCoens

Differential Revision: D6597628

Pulled By: patapizza

fbshipit-source-id: 8bac0f686d6cecc38d9998e37042fe48f73530dc
2017-12-19 13:15:30 -08:00
Henri Dwyer
15b3988ec5 update MY pattern match to HashMap
Summary: Create hashmaps for the pattern matching rules, and make the regex matching read the value from the hashmap.

Reviewed By: patapizza

Differential Revision: D6592508

fbshipit-source-id: 0f4dd9bc548f71d0e6e52c938bd2e39bfd909c01
2017-12-18 13:00:35 -08:00
Henri Dwyer
10db72fd05 update KO pattern match to HashMap
Summary: Create hashmaps for the pattern matching rules, and make the regex matching read the value from the hashmap.

Reviewed By: panagosg7

Differential Revision: D6536790

fbshipit-source-id: ecb0bf28cf22b82ef10c3c2408eda15409189431
2017-12-15 11:45:28 -08:00
Julien Odent
fb3979d257 Numeral: move ruleFractions to common + cleanup numeral flag for Time
Summary:
* Closes PRs #115, #116.
* `Numeral`: Moves `ruleFractions` to common
* `Numeral/PT`: fixes dot in regex
* `Time`: Moves `isNumeralSafeToUse` to `isIntegerBetween` (more robust and cleaner)
* `Time/EN`: refactored `ruleCycleLastNextN` to `ruleDurationLastNext` (so it also accounts for non-safe Numerals, like "in a few days")

Reviewed By: blandinw

Differential Revision: D6502958

fbshipit-source-id: 6d9c08a23dab88f441aadade08d663c8e0da415d
2017-12-15 10:15:31 -08:00
Julien Odent
6df3b26707 Numeral: common rule + supporting hindu-arabic numerals for Burmese
Summary:
* `ruleIntegerNumeric` was used in all languages but Burmese.
* it seems like the hindu-arabic numerals are slowly getting in Burmese (e.g. recent car plates)
* Moving the rule in `Duckling/Numeral/Common.hs`

Reviewed By: blandinw

Differential Revision: D6498349

fbshipit-source-id: e868dc9960f18f0781e4aa98a0dfcd14969537c9
2017-12-06 16:00:28 -08:00