Commit Graph

740 Commits

Author SHA1 Message Date
yasen-yankov
21c3b32e4d Added TimeGrain, Duration, and Ordinal
Summary: Closes https://github.com/facebook/duckling/pull/164

Reviewed By: chinmay87

Differential Revision: D7280110

Pulled By: patapizza

fbshipit-source-id: d98ddd900fe83f06b28afd39ea3311f42716288c
2018-03-19 18:00:29 -07:00
Giri Anantharaman
519c9519a3 Support Tamil numerals
Summary:
* Setup Tamil (TA) language
* Added Numeral Dimension

Reviewed By: patapizza

Differential Revision: D7323636

fbshipit-source-id: 4b1a42197ff4799880cded9ce86b8d7fae1507bc
2018-03-19 16:45:36 -07:00
Chinmay Deshmukh
5ac990bbe2 Return latent entities
Summary: Add an option to return latent time entities. This can be used when one is pretty certain that the input contains a datetime.

Reviewed By: patapizza

Differential Revision: D7254245

fbshipit-source-id: e9e0503cace2691804056fcebdc18fd9090fb181
2018-03-19 14:45:27 -07:00
Julien Odent
490799fe4a Time/EN: Periodic holidays
Summary:
* added periodic holidays for all existing English locales
* moved Independence Day from common to US, as it is country-dependent
* moved Memorial Day from common to US, as it is country-dependent (fixed the one for Canada)
* added Decoration Day as an alias for US Memorial Day

Reviewed By: chinmay87

Differential Revision: D7294129

fbshipit-source-id: 89cdbf91b4c43e1996fa1f4509eac1d1b1978197
2018-03-16 17:17:08 -07:00
yasen-yankov
1493d44465 Added BG distance
Summary: Closes https://github.com/facebook/duckling/pull/162

Reviewed By: chinmay87

Differential Revision: D7239243

Pulled By: patapizza

fbshipit-source-id: a5518219a67fa46bb06e97eee3dfd07ab683162f
2018-03-13 15:45:32 -07:00
Julien Odent
af0751a748 Time/EN: handle Easter
Summary:
* introducing `Duckling.Time.Computed` for pre-computed days
* handle Easter Sunday and Easter Monday from 1950 to 2050

Reviewed By: JonCoens

Differential Revision: D7207643

fbshipit-source-id: 13e1d6d10dc9cd5d18ef7ff0c50d99e695c3cb1f
2018-03-13 10:15:50 -07:00
Julien Odent
4c3af007d5 0.1.4.0 cut
Summary: Bumping version before Hackage release.

Reviewed By: JonCoens

Differential Revision: D7240076

fbshipit-source-id: 063d4acb9e9e689c786a20aaf522f35b3cd56483
2018-03-12 11:00:57 -07:00
Julien Odent
ad0ab1d98e Time/DE,EN,NL: fixes + refactoring
Summary:
* added `okForThisNext` flag for holidays/seasons/months/days of week rules
* fixed `mkRuleHolidays` argument ordering
* reused helpers in English

Reviewed By: JonCoens

Differential Revision: D7200945

fbshipit-source-id: 01da94f5ba929a37a379b5b9efd68961892a926c
2018-03-12 09:45:37 -07:00
Ezgi Çiçek
2254034e62 Add Node to Entity to pass along the parse tree information
Summary:
- add `Node` field to `Entity`
- ignore `Node` field of `Entity` for toJSON for now (will be fixed later)
- change `Debug.hs` so that we print the respective Entity's Node
- add wildcard for the new Node field in `parseTest` of Api/Tests.hs`

Reviewed By: patapizza

Differential Revision: D7174696

fbshipit-source-id: 240e4c53b72323b500ac58a74f873ce247bb3387
2018-03-07 10:45:28 -08:00
Panagiotis Vekris
169df6b46b ruleSkipHundreds only accepts alphabetic numerals
Summary:
Before, duckling would parse `"1 2 2"` as the three digit number 122 through
`ruleSkipHundreds`. This, however, allowed the string `"Pay Kiran1 10eur"` to be
parsed as "110 EUR", which was reported in https://github.com/facebook/duckling/issues/159.

This change only accepts alphabetic numerals to pass through `ruleSkipHundreds`
(for example `"one twenty two"`), which presumably was the original intention of
this rule. This fixes the above issue, without any change in Corpus.

Reviewed By: patapizza

Differential Revision: D7151934

fbshipit-source-id: 024a7a0b6a53beb3a0d42d4bb7f542ce3b05726b
2018-03-06 17:00:29 -08:00
Mateusz Kwapich
c875ffbce6 test commit
fbshipit-source-id: 42e3e4df7f8ea21241b5706338d8f3b456e4cf05
2018-03-06 05:00:29 -08:00
Mateusz Kwapich
ed7eeb4cad test commit
fbshipit-source-id: 7b1864d58dbcf5454b8576ff1e4df6aef93f7647
2018-03-06 05:00:29 -08:00
Bora Tunca
002dec5614 Support lowercase time zones
Summary: Support lowercase time zones

Reviewed By: patapizza

Differential Revision: D7114137

fbshipit-source-id: 906c4fa5305aea8990e8a5d480ceeec4b584d7db
2018-03-01 14:30:29 -08:00
Chris Reid
b637deedc7 Add EN Quantity Support Intervals
Summary: Added support for quantity intervals (eg. between, above, below) and EN rules.

Reviewed By: patapizza

Differential Revision: D6985854

fbshipit-source-id: 1c63e14c66f5a6391960270e232aa10bf87307b4
2018-02-22 09:30:29 -08:00
Julien Odent
aed5b8f779 Numeral: don't compose negative numbers, fix double negatives
Summary:
* added `isMultipliable` helper and used that in patterns along with `isPositive`
* fixed double negatives in most languages

Reviewed By: niteria

Differential Revision: D7034982

fbshipit-source-id: a0bb67056d3107167830ece0c34d761c5563c5a7
2018-02-21 12:02:45 -08:00
Julien Odent
55b0d1d437 Distance: Don't return type in interval bounds
Summary: We were returning `"type":"value"` for each bound.

Reviewed By: blandinw

Differential Revision: D7011092

fbshipit-source-id: d2b5c4d28505a1f24749ecc3975e04c7ba7a6fe1
2018-02-16 11:30:32 -08:00
bidhan-a
43079e7113 Setup Nepali (NE) and add Numeral dimension
Summary:
- Setup Nepali (NE) language
- Add basic Numeral dimension
Closes https://github.com/facebook/duckling/pull/156

Reviewed By: JonCoens

Differential Revision: D6965558

Pulled By: patapizza

fbshipit-source-id: f46c9b104d4345f20bd0cf53f8c9c8754855f314
2018-02-13 07:45:31 -08:00
Julien Odent
c7fb533a67 Url: handle fragments
Summary: https://github.com/facebook/duckling/issues/155

Reviewed By: JonCoens

Differential Revision: D6967145

fbshipit-source-id: 44e573b57c714a62d93d74063f597d31feb8c6f5
2018-02-12 13:00:29 -08:00
Lucas Hosseini
1f7290880c Distance interval support.
Summary: Interval support for distances (common types + EN rules).

Reviewed By: patapizza

Differential Revision: D6942340

fbshipit-source-id: d05ff337167b11c9186f38ca04bacf0d24a41526
2018-02-09 16:16:04 -08:00
Ashwini Reddy Challa
c8501d3e85 Added HI Ordinals
Summary: Added ordinal support for Hindi

Reviewed By: patapizza

Differential Revision: D6944156

fbshipit-source-id: eb5da698e5cccde9a1cc31adf7bc433b89e07454
2018-02-09 13:15:30 -08:00
Abdallatif Sulaiman
0c395f7a1b fix Valentine's Day regex in arabic
Summary: Closes https://github.com/facebook/duckling/pull/146

Reviewed By: adelnobel

Differential Revision: D6834246

Pulled By: patapizza

fbshipit-source-id: ba90be16e877f9a39162820f1071ce7a6d6da6d7
2018-02-05 14:30:34 -08:00
Panagiotis Vekris
7a89ced6db Intervals for AmountOfMoney
Summary:
Adds support for intervals for amounts of money, e.g. "entre 12 euros et 15 euros"

Addresses https://github.com/facebook/duckling/issues/151

Reviewed By: patapizza

Differential Revision: D6870091

fbshipit-source-id: 750815129159e75323d00aa56fdda898baa32fdc
2018-02-02 17:15:28 -08:00
Bill Cromie
197fef1b5e Ordinals with spaces, en-dashes and more!
Summary:
Adding more support for "natural" language compound ordinals:
"twenty nine", "twentynine", and "twenty–nine".

These tend to come up in text that has been transcribed from audio, informal user input, and more formal printed material respectively.
Closes https://github.com/facebook/duckling/pull/152

Reviewed By: panagosg7

Differential Revision: D6850514

Pulled By: patapizza

fbshipit-source-id: 1d3491655cfc1fb3faab0c3e196de1b2361ae95e
2018-02-01 14:30:44 -08:00
Julien Odent
b00fa512af AmountOfMoney: fixes
Summary:
* don't recursively compose cents
* don't allow decreasing ranges

Reviewed By: blandinw

Differential Revision: D6849132

fbshipit-source-id: ed6ca30388642c21e677a628971747a4fb3dfbef
2018-01-30 14:45:36 -08:00
Julien Odent
bef7a44fa8 Remove redundant brackets and language pragmas
Summary: .

Reviewed By: JonCoens

Differential Revision: D6838082

fbshipit-source-id: 94757bdb80c6d3c29a7a6554429940a1b7403108
2018-01-29 16:45:28 -08:00
Filipe Pereira
81a97575f6 PR for issue #148
Summary:
patapizza Did my best, but I'm neither fluent on Dutch or Haskell :) I tried keeping it consistent with the existing code, let me know if there's something I can improve!
Closes https://github.com/facebook/duckling/pull/149

Reviewed By: blandinw

Differential Revision: D6809767

Pulled By: patapizza

fbshipit-source-id: f92802b94ed7588a03219f63e74dbab475e950ab
2018-01-26 09:46:04 -08:00
Magnus Burton
7d864e38ab Added more synonyms to about <duration>
Summary:
Added some more synonyms to the duration part of the Swedish Duckling. This will match even more **valid** durations that match `about <duration>`.
Closes https://github.com/facebook/duckling/pull/145

Differential Revision: D6775860

Pulled By: patapizza

fbshipit-source-id: d0c844747e3bbb60c2500fb4b79aa53c2680ab48
2018-01-22 17:02:16 -08:00
vxovka
b964d02a7d Russian ordinals fixes and additions
Summary:
Fixed 3rd ordinal which didn't work for feminine and neuter genders in nominative.
Fixed when ordinals from 1 to 19 not recognized when used with capital letters.
Added support for ordinals in genitive for all genders.
Closes https://github.com/facebook/duckling/pull/141

Reviewed By: patapizza

Differential Revision: D6740190

Pulled By: panagosg7

fbshipit-source-id: a6a676d95890a7e156346f479c84a71c1b2bd17f
2018-01-21 23:00:41 -08:00
Julien Odent
7f21d1d7d7 Time/EN: Parse MM.DD.YYYY (or DD.MM.YYYY)
Summary:
Fixes #143.

Doesn't allow for MM.DD.YY (or DD.MM.YY) as this clashes with HHMMSS.

Reviewed By: blandinw

Differential Revision: D6765534

fbshipit-source-id: 1d1720e5bb292fdb3970e76475f35eab316048a8
2018-01-19 19:15:49 -08:00
Akshay Patil
d590f098dc Add type for Duration
Summary: Added missing `type` for Duration.

Reviewed By: patapizza

Differential Revision: D6717680

fbshipit-source-id: d1454758a3852bb5c06fb7a14224012cb3085a58
2018-01-16 15:15:27 -08:00
Julien Odent
da36ab8a80 Time: fix empty values for past time
Summary: "yesterday" would resolve to an entity without any `values`.

Reviewed By: JonCoens

Differential Revision: D6697432

fbshipit-source-id: 7b15727f92703842a2995210fdeb99c00be74bc3
2018-01-10 15:30:32 -08:00
Panagiotis Vekris
f28ad7a367 support for reference time param in web server
Summary:
PR for the issue #132
Closes https://github.com/facebook/duckling/pull/133

Reviewed By: patapizza

Differential Revision: D6661000

Pulled By: panagosg7

fbshipit-source-id: da061cb55e86c8747ff885df9881e270d22cdfa7
2018-01-10 12:45:37 -08:00
Julien Odent
ffdb976dd4 Time/SV: Add more examples
Summary: #140.

Reviewed By: panagosg7

Differential Revision: D6667848

fbshipit-source-id: 861fb87caf10cac64f0fe2eaa5186a70da6a3245
2018-01-05 12:15:40 -08:00
Filipe Pereira
9f78ebd801 PR for issue #136
Summary: Closes https://github.com/facebook/duckling/pull/139

Reviewed By: panagosg7

Differential Revision: D6661224

Pulled By: patapizza

fbshipit-source-id: 4802c2f66efd017577f3247e3b4abddcb7496c3f
2018-01-05 12:15:40 -08:00
Abdallatif Sulaiman
ce2f48fbb5 Quickfix for Arabic Numeral
Summary: Closes https://github.com/facebook/duckling/pull/137

Reviewed By: adelnobel

Differential Revision: D6660959

Pulled By: patapizza

fbshipit-source-id: 51169c63558b5a35f16bdaaada9649c88a4d2994
2018-01-05 11:45:40 -08:00
Mark Neumann
92e321e9ad Add note about -j1 flag for stack build
Summary:
I had trouble building the Docker image on a mac because `stack build` was trying to use all the cores and subsequently running out of memory. It took me a while to find the right flag, so I added a note to the dockerfile.
Closes https://github.com/facebook/duckling/pull/135

Differential Revision: D6655605

Pulled By: patapizza

fbshipit-source-id: 559c9fde175ad89f64b5c487afb77e3b7d5b982f
2018-01-03 06:00:44 -08:00
Abdallatif Sulaiman
89822776c6 Added Amount of Money Dimension to Arabic language
Summary: Closes https://github.com/facebook/duckling/pull/134

Reviewed By: JonCoens

Differential Revision: D6649920

Pulled By: patapizza

fbshipit-source-id: 9c647c84f5ae4f3dc26cb0c7aa74abb097ea001a
2018-01-02 14:00:51 -08:00
Jade Auer
fc574a448e add FB code of conduct
Summary:
Add FB-standard code of conduct for open source projects.
ref: https://fburl.com/aedakn3u
Closes https://github.com/facebook/duckling/pull/128

Reviewed By: patapizza

Differential Revision: D6629244

Pulled By: jda

fbshipit-source-id: 50ef889052f97757220157d68dc531f5e72bf86f
2018-01-02 08:15:29 -08:00
Abdallatif Sulaiman
2d726d3837 Added Quantity Dimension to Arabic language
Summary: Closes https://github.com/facebook/duckling/pull/127

Reviewed By: panagosg7

Differential Revision: D6616298

Pulled By: patapizza

fbshipit-source-id: dc774ff1e870bbd083a9cca8ee6f75db852afce9
2017-12-20 18:00:43 -08:00
Abdallatif Sulaiman
c056a0b46a Added Temperature Dimension to Arabic language
Summary: Closes https://github.com/facebook/duckling/pull/126

Reviewed By: panagosg7

Differential Revision: D6614995

Pulled By: patapizza

fbshipit-source-id: 35ff142a3fc8a498d9abbb80d28dc9be6cdcbc4d
2017-12-20 18:00:43 -08:00
Abdallatif Sulaiman
380457db8f Added Volume Dimension to Arabic language
Summary: Closes https://github.com/facebook/duckling/pull/125

Reviewed By: panagosg7

Differential Revision: D6614574

Pulled By: patapizza

fbshipit-source-id: 054ed04e0cc1cf79be31340ebb8b7fea3bc67f57
2017-12-20 18:00:43 -08:00
Abdallatif Sulaiman
1393098bcc Added Time Dimension to Arabic
Summary:
Hi, in this pr:
* Added time dimension to Arabic language, thanks to Hussein-Dahir & Yazeed-Obaid for writing time corpus.
* Fixed some bugs in numeral & ordinals and added more test cases for them.

Also, I don't really understand why do we use classifiers in time dimension?
Closes https://github.com/facebook/duckling/pull/123

Reviewed By: blandinw

Differential Revision: D6583313

Pulled By: patapizza

fbshipit-source-id: f7acdef0c032d7b7fd7d224832fdaf484d2df825
2017-12-19 14:30:42 -08:00
Newinfinite007
c133bad24a Hindi Language Numeral Dimension(minimalistic model). Tests passed.
Summary: Closes https://github.com/facebook/duckling/pull/119

Reviewed By: JonCoens

Differential Revision: D6597628

Pulled By: patapizza

fbshipit-source-id: 8bac0f686d6cecc38d9998e37042fe48f73530dc
2017-12-19 13:15:30 -08:00
Henri Dwyer
15b3988ec5 update MY pattern match to HashMap
Summary: Create hashmaps for the pattern matching rules, and make the regex matching read the value from the hashmap.

Reviewed By: patapizza

Differential Revision: D6592508

fbshipit-source-id: 0f4dd9bc548f71d0e6e52c938bd2e39bfd909c01
2017-12-18 13:00:35 -08:00
Henri Dwyer
10db72fd05 update KO pattern match to HashMap
Summary: Create hashmaps for the pattern matching rules, and make the regex matching read the value from the hashmap.

Reviewed By: panagosg7

Differential Revision: D6536790

fbshipit-source-id: ecb0bf28cf22b82ef10c3c2408eda15409189431
2017-12-15 11:45:28 -08:00
Julien Odent
fb3979d257 Numeral: move ruleFractions to common + cleanup numeral flag for Time
Summary:
* Closes PRs #115, #116.
* `Numeral`: Moves `ruleFractions` to common
* `Numeral/PT`: fixes dot in regex
* `Time`: Moves `isNumeralSafeToUse` to `isIntegerBetween` (more robust and cleaner)
* `Time/EN`: refactored `ruleCycleLastNextN` to `ruleDurationLastNext` (so it also accounts for non-safe Numerals, like "in a few days")

Reviewed By: blandinw

Differential Revision: D6502958

fbshipit-source-id: 6d9c08a23dab88f441aadade08d663c8e0da415d
2017-12-15 10:15:31 -08:00
Zahar Shimanchik
2274e40369 Added AmountOfMoney Dimension to Russian language
Summary: Closes https://github.com/facebook/duckling/pull/120

Reviewed By: patapizza

Differential Revision: D6520302

Pulled By: panagosg7

fbshipit-source-id: 2c9ac6e15ca3ee90b2bf5911ee62835966fcacd1
2017-12-12 12:15:30 -08:00
Julien Odent
6df3b26707 Numeral: common rule + supporting hindu-arabic numerals for Burmese
Summary:
* `ruleIntegerNumeric` was used in all languages but Burmese.
* it seems like the hindu-arabic numerals are slowly getting in Burmese (e.g. recent car plates)
* Moving the rule in `Duckling/Numeral/Common.hs`

Reviewed By: blandinw

Differential Revision: D6498349

fbshipit-source-id: e868dc9960f18f0781e4aa98a0dfcd14969537c9
2017-12-06 16:00:28 -08:00
Genki Furumi
56b57df153 HashMap lookups for large regexes
Summary: Replaced pattern matching with Hashmap.

Reviewed By: patapizza

Differential Revision: D6493427

fbshipit-source-id: 5ae9370387a738423724cabbe7c0d44c4889e185
2017-12-06 11:00:38 -08:00
Alex Torres
498e8b16e6 fix pt rules for numeral
Summary:
When I write "dois mil e duzentos" the result should be 2200, but duckling recognize the numbers separated and give the result:

`[{"dim":"number","body":"dois","value":{"value":2,"type":"value"},"start":0,"end":4},{"dim":"number","body":"mil","value":{"value":1000,"type":"value"},"start":5,"end":8},{"dim":"time","body":"mil","value":{"values":[],"value":"1000-01-01T00:00:00.000-07:53","grain":"year","type":"value"},"start":5,"end":8},{"dim":"number","body":"duzentos","value":{"value":200,"type":"value"},"start":11,"end":19}]`

Now with this commit, duckling gives the correct result:

`[{"dim":"number","body":"dois mil e duzentos","value":{"value":2200,"type":"value"},"start":0,"end":19}]`
Closes https://github.com/facebook/duckling/pull/117

Reviewed By: blandinw

Differential Revision: D6477925

Pulled By: patapizza

fbshipit-source-id: 26ab503cc8def739c51ceb5bae7546016ba65ad6
2017-12-04 18:00:36 -08:00