Commit Graph

238 Commits

Author SHA1 Message Date
yasen-yankov
1493d44465 Added BG distance
Summary: Closes https://github.com/facebook/duckling/pull/162

Reviewed By: chinmay87

Differential Revision: D7239243

Pulled By: patapizza

fbshipit-source-id: a5518219a67fa46bb06e97eee3dfd07ab683162f
2018-03-13 15:45:32 -07:00
Julien Odent
af0751a748 Time/EN: handle Easter
Summary:
* introducing `Duckling.Time.Computed` for pre-computed days
* handle Easter Sunday and Easter Monday from 1950 to 2050

Reviewed By: JonCoens

Differential Revision: D7207643

fbshipit-source-id: 13e1d6d10dc9cd5d18ef7ff0c50d99e695c3cb1f
2018-03-13 10:15:50 -07:00
Julien Odent
ad0ab1d98e Time/DE,EN,NL: fixes + refactoring
Summary:
* added `okForThisNext` flag for holidays/seasons/months/days of week rules
* fixed `mkRuleHolidays` argument ordering
* reused helpers in English

Reviewed By: JonCoens

Differential Revision: D7200945

fbshipit-source-id: 01da94f5ba929a37a379b5b9efd68961892a926c
2018-03-12 09:45:37 -07:00
Ezgi Çiçek
2254034e62 Add Node to Entity to pass along the parse tree information
Summary:
- add `Node` field to `Entity`
- ignore `Node` field of `Entity` for toJSON for now (will be fixed later)
- change `Debug.hs` so that we print the respective Entity's Node
- add wildcard for the new Node field in `parseTest` of Api/Tests.hs`

Reviewed By: patapizza

Differential Revision: D7174696

fbshipit-source-id: 240e4c53b72323b500ac58a74f873ce247bb3387
2018-03-07 10:45:28 -08:00
Panagiotis Vekris
169df6b46b ruleSkipHundreds only accepts alphabetic numerals
Summary:
Before, duckling would parse `"1 2 2"` as the three digit number 122 through
`ruleSkipHundreds`. This, however, allowed the string `"Pay Kiran1 10eur"` to be
parsed as "110 EUR", which was reported in https://github.com/facebook/duckling/issues/159.

This change only accepts alphabetic numerals to pass through `ruleSkipHundreds`
(for example `"one twenty two"`), which presumably was the original intention of
this rule. This fixes the above issue, without any change in Corpus.

Reviewed By: patapizza

Differential Revision: D7151934

fbshipit-source-id: 024a7a0b6a53beb3a0d42d4bb7f542ce3b05726b
2018-03-06 17:00:29 -08:00
Bora Tunca
002dec5614 Support lowercase time zones
Summary: Support lowercase time zones

Reviewed By: patapizza

Differential Revision: D7114137

fbshipit-source-id: 906c4fa5305aea8990e8a5d480ceeec4b584d7db
2018-03-01 14:30:29 -08:00
Chris Reid
b637deedc7 Add EN Quantity Support Intervals
Summary: Added support for quantity intervals (eg. between, above, below) and EN rules.

Reviewed By: patapizza

Differential Revision: D6985854

fbshipit-source-id: 1c63e14c66f5a6391960270e232aa10bf87307b4
2018-02-22 09:30:29 -08:00
Julien Odent
aed5b8f779 Numeral: don't compose negative numbers, fix double negatives
Summary:
* added `isMultipliable` helper and used that in patterns along with `isPositive`
* fixed double negatives in most languages

Reviewed By: niteria

Differential Revision: D7034982

fbshipit-source-id: a0bb67056d3107167830ece0c34d761c5563c5a7
2018-02-21 12:02:45 -08:00
Julien Odent
55b0d1d437 Distance: Don't return type in interval bounds
Summary: We were returning `"type":"value"` for each bound.

Reviewed By: blandinw

Differential Revision: D7011092

fbshipit-source-id: d2b5c4d28505a1f24749ecc3975e04c7ba7a6fe1
2018-02-16 11:30:32 -08:00
bidhan-a
43079e7113 Setup Nepali (NE) and add Numeral dimension
Summary:
- Setup Nepali (NE) language
- Add basic Numeral dimension
Closes https://github.com/facebook/duckling/pull/156

Reviewed By: JonCoens

Differential Revision: D6965558

Pulled By: patapizza

fbshipit-source-id: f46c9b104d4345f20bd0cf53f8c9c8754855f314
2018-02-13 07:45:31 -08:00
Julien Odent
c7fb533a67 Url: handle fragments
Summary: https://github.com/facebook/duckling/issues/155

Reviewed By: JonCoens

Differential Revision: D6967145

fbshipit-source-id: 44e573b57c714a62d93d74063f597d31feb8c6f5
2018-02-12 13:00:29 -08:00
Lucas Hosseini
1f7290880c Distance interval support.
Summary: Interval support for distances (common types + EN rules).

Reviewed By: patapizza

Differential Revision: D6942340

fbshipit-source-id: d05ff337167b11c9186f38ca04bacf0d24a41526
2018-02-09 16:16:04 -08:00
Ashwini Reddy Challa
c8501d3e85 Added HI Ordinals
Summary: Added ordinal support for Hindi

Reviewed By: patapizza

Differential Revision: D6944156

fbshipit-source-id: eb5da698e5cccde9a1cc31adf7bc433b89e07454
2018-02-09 13:15:30 -08:00
Abdallatif Sulaiman
0c395f7a1b fix Valentine's Day regex in arabic
Summary: Closes https://github.com/facebook/duckling/pull/146

Reviewed By: adelnobel

Differential Revision: D6834246

Pulled By: patapizza

fbshipit-source-id: ba90be16e877f9a39162820f1071ce7a6d6da6d7
2018-02-05 14:30:34 -08:00
Panagiotis Vekris
7a89ced6db Intervals for AmountOfMoney
Summary:
Adds support for intervals for amounts of money, e.g. "entre 12 euros et 15 euros"

Addresses https://github.com/facebook/duckling/issues/151

Reviewed By: patapizza

Differential Revision: D6870091

fbshipit-source-id: 750815129159e75323d00aa56fdda898baa32fdc
2018-02-02 17:15:28 -08:00
Bill Cromie
197fef1b5e Ordinals with spaces, en-dashes and more!
Summary:
Adding more support for "natural" language compound ordinals:
"twenty nine", "twentynine", and "twenty–nine".

These tend to come up in text that has been transcribed from audio, informal user input, and more formal printed material respectively.
Closes https://github.com/facebook/duckling/pull/152

Reviewed By: panagosg7

Differential Revision: D6850514

Pulled By: patapizza

fbshipit-source-id: 1d3491655cfc1fb3faab0c3e196de1b2361ae95e
2018-02-01 14:30:44 -08:00
Julien Odent
b00fa512af AmountOfMoney: fixes
Summary:
* don't recursively compose cents
* don't allow decreasing ranges

Reviewed By: blandinw

Differential Revision: D6849132

fbshipit-source-id: ed6ca30388642c21e677a628971747a4fb3dfbef
2018-01-30 14:45:36 -08:00
Julien Odent
bef7a44fa8 Remove redundant brackets and language pragmas
Summary: .

Reviewed By: JonCoens

Differential Revision: D6838082

fbshipit-source-id: 94757bdb80c6d3c29a7a6554429940a1b7403108
2018-01-29 16:45:28 -08:00
Filipe Pereira
81a97575f6 PR for issue #148
Summary:
patapizza Did my best, but I'm neither fluent on Dutch or Haskell :) I tried keeping it consistent with the existing code, let me know if there's something I can improve!
Closes https://github.com/facebook/duckling/pull/149

Reviewed By: blandinw

Differential Revision: D6809767

Pulled By: patapizza

fbshipit-source-id: f92802b94ed7588a03219f63e74dbab475e950ab
2018-01-26 09:46:04 -08:00
Magnus Burton
7d864e38ab Added more synonyms to about <duration>
Summary:
Added some more synonyms to the duration part of the Swedish Duckling. This will match even more **valid** durations that match `about <duration>`.
Closes https://github.com/facebook/duckling/pull/145

Differential Revision: D6775860

Pulled By: patapizza

fbshipit-source-id: d0c844747e3bbb60c2500fb4b79aa53c2680ab48
2018-01-22 17:02:16 -08:00
vxovka
b964d02a7d Russian ordinals fixes and additions
Summary:
Fixed 3rd ordinal which didn't work for feminine and neuter genders in nominative.
Fixed when ordinals from 1 to 19 not recognized when used with capital letters.
Added support for ordinals in genitive for all genders.
Closes https://github.com/facebook/duckling/pull/141

Reviewed By: patapizza

Differential Revision: D6740190

Pulled By: panagosg7

fbshipit-source-id: a6a676d95890a7e156346f479c84a71c1b2bd17f
2018-01-21 23:00:41 -08:00
Julien Odent
7f21d1d7d7 Time/EN: Parse MM.DD.YYYY (or DD.MM.YYYY)
Summary:
Fixes #143.

Doesn't allow for MM.DD.YY (or DD.MM.YY) as this clashes with HHMMSS.

Reviewed By: blandinw

Differential Revision: D6765534

fbshipit-source-id: 1d1720e5bb292fdb3970e76475f35eab316048a8
2018-01-19 19:15:49 -08:00
Akshay Patil
d590f098dc Add type for Duration
Summary: Added missing `type` for Duration.

Reviewed By: patapizza

Differential Revision: D6717680

fbshipit-source-id: d1454758a3852bb5c06fb7a14224012cb3085a58
2018-01-16 15:15:27 -08:00
Julien Odent
da36ab8a80 Time: fix empty values for past time
Summary: "yesterday" would resolve to an entity without any `values`.

Reviewed By: JonCoens

Differential Revision: D6697432

fbshipit-source-id: 7b15727f92703842a2995210fdeb99c00be74bc3
2018-01-10 15:30:32 -08:00
Julien Odent
ffdb976dd4 Time/SV: Add more examples
Summary: #140.

Reviewed By: panagosg7

Differential Revision: D6667848

fbshipit-source-id: 861fb87caf10cac64f0fe2eaa5186a70da6a3245
2018-01-05 12:15:40 -08:00
Filipe Pereira
9f78ebd801 PR for issue #136
Summary: Closes https://github.com/facebook/duckling/pull/139

Reviewed By: panagosg7

Differential Revision: D6661224

Pulled By: patapizza

fbshipit-source-id: 4802c2f66efd017577f3247e3b4abddcb7496c3f
2018-01-05 12:15:40 -08:00
Abdallatif Sulaiman
ce2f48fbb5 Quickfix for Arabic Numeral
Summary: Closes https://github.com/facebook/duckling/pull/137

Reviewed By: adelnobel

Differential Revision: D6660959

Pulled By: patapizza

fbshipit-source-id: 51169c63558b5a35f16bdaaada9649c88a4d2994
2018-01-05 11:45:40 -08:00
Abdallatif Sulaiman
89822776c6 Added Amount of Money Dimension to Arabic language
Summary: Closes https://github.com/facebook/duckling/pull/134

Reviewed By: JonCoens

Differential Revision: D6649920

Pulled By: patapizza

fbshipit-source-id: 9c647c84f5ae4f3dc26cb0c7aa74abb097ea001a
2018-01-02 14:00:51 -08:00
Abdallatif Sulaiman
2d726d3837 Added Quantity Dimension to Arabic language
Summary: Closes https://github.com/facebook/duckling/pull/127

Reviewed By: panagosg7

Differential Revision: D6616298

Pulled By: patapizza

fbshipit-source-id: dc774ff1e870bbd083a9cca8ee6f75db852afce9
2017-12-20 18:00:43 -08:00
Abdallatif Sulaiman
c056a0b46a Added Temperature Dimension to Arabic language
Summary: Closes https://github.com/facebook/duckling/pull/126

Reviewed By: panagosg7

Differential Revision: D6614995

Pulled By: patapizza

fbshipit-source-id: 35ff142a3fc8a498d9abbb80d28dc9be6cdcbc4d
2017-12-20 18:00:43 -08:00
Abdallatif Sulaiman
380457db8f Added Volume Dimension to Arabic language
Summary: Closes https://github.com/facebook/duckling/pull/125

Reviewed By: panagosg7

Differential Revision: D6614574

Pulled By: patapizza

fbshipit-source-id: 054ed04e0cc1cf79be31340ebb8b7fea3bc67f57
2017-12-20 18:00:43 -08:00
Abdallatif Sulaiman
1393098bcc Added Time Dimension to Arabic
Summary:
Hi, in this pr:
* Added time dimension to Arabic language, thanks to Hussein-Dahir & Yazeed-Obaid for writing time corpus.
* Fixed some bugs in numeral & ordinals and added more test cases for them.

Also, I don't really understand why do we use classifiers in time dimension?
Closes https://github.com/facebook/duckling/pull/123

Reviewed By: blandinw

Differential Revision: D6583313

Pulled By: patapizza

fbshipit-source-id: f7acdef0c032d7b7fd7d224832fdaf484d2df825
2017-12-19 14:30:42 -08:00
Newinfinite007
c133bad24a Hindi Language Numeral Dimension(minimalistic model). Tests passed.
Summary: Closes https://github.com/facebook/duckling/pull/119

Reviewed By: JonCoens

Differential Revision: D6597628

Pulled By: patapizza

fbshipit-source-id: 8bac0f686d6cecc38d9998e37042fe48f73530dc
2017-12-19 13:15:30 -08:00
Henri Dwyer
15b3988ec5 update MY pattern match to HashMap
Summary: Create hashmaps for the pattern matching rules, and make the regex matching read the value from the hashmap.

Reviewed By: patapizza

Differential Revision: D6592508

fbshipit-source-id: 0f4dd9bc548f71d0e6e52c938bd2e39bfd909c01
2017-12-18 13:00:35 -08:00
Henri Dwyer
10db72fd05 update KO pattern match to HashMap
Summary: Create hashmaps for the pattern matching rules, and make the regex matching read the value from the hashmap.

Reviewed By: panagosg7

Differential Revision: D6536790

fbshipit-source-id: ecb0bf28cf22b82ef10c3c2408eda15409189431
2017-12-15 11:45:28 -08:00
Julien Odent
fb3979d257 Numeral: move ruleFractions to common + cleanup numeral flag for Time
Summary:
* Closes PRs #115, #116.
* `Numeral`: Moves `ruleFractions` to common
* `Numeral/PT`: fixes dot in regex
* `Time`: Moves `isNumeralSafeToUse` to `isIntegerBetween` (more robust and cleaner)
* `Time/EN`: refactored `ruleCycleLastNextN` to `ruleDurationLastNext` (so it also accounts for non-safe Numerals, like "in a few days")

Reviewed By: blandinw

Differential Revision: D6502958

fbshipit-source-id: 6d9c08a23dab88f441aadade08d663c8e0da415d
2017-12-15 10:15:31 -08:00
Zahar Shimanchik
2274e40369 Added AmountOfMoney Dimension to Russian language
Summary: Closes https://github.com/facebook/duckling/pull/120

Reviewed By: patapizza

Differential Revision: D6520302

Pulled By: panagosg7

fbshipit-source-id: 2c9ac6e15ca3ee90b2bf5911ee62835966fcacd1
2017-12-12 12:15:30 -08:00
Julien Odent
6df3b26707 Numeral: common rule + supporting hindu-arabic numerals for Burmese
Summary:
* `ruleIntegerNumeric` was used in all languages but Burmese.
* it seems like the hindu-arabic numerals are slowly getting in Burmese (e.g. recent car plates)
* Moving the rule in `Duckling/Numeral/Common.hs`

Reviewed By: blandinw

Differential Revision: D6498349

fbshipit-source-id: e868dc9960f18f0781e4aa98a0dfcd14969537c9
2017-12-06 16:00:28 -08:00
Genki Furumi
56b57df153 HashMap lookups for large regexes
Summary: Replaced pattern matching with Hashmap.

Reviewed By: patapizza

Differential Revision: D6493427

fbshipit-source-id: 5ae9370387a738423724cabbe7c0d44c4889e185
2017-12-06 11:00:38 -08:00
Alex Torres
498e8b16e6 fix pt rules for numeral
Summary:
When I write "dois mil e duzentos" the result should be 2200, but duckling recognize the numbers separated and give the result:

`[{"dim":"number","body":"dois","value":{"value":2,"type":"value"},"start":0,"end":4},{"dim":"number","body":"mil","value":{"value":1000,"type":"value"},"start":5,"end":8},{"dim":"time","body":"mil","value":{"values":[],"value":"1000-01-01T00:00:00.000-07:53","grain":"year","type":"value"},"start":5,"end":8},{"dim":"number","body":"duzentos","value":{"value":200,"type":"value"},"start":11,"end":19}]`

Now with this commit, duckling gives the correct result:

`[{"dim":"number","body":"dois mil e duzentos","value":{"value":2200,"type":"value"},"start":0,"end":19}]`
Closes https://github.com/facebook/duckling/pull/117

Reviewed By: blandinw

Differential Revision: D6477925

Pulled By: patapizza

fbshipit-source-id: 26ab503cc8def739c51ceb5bae7546016ba65ad6
2017-12-04 18:00:36 -08:00
Panagiotis Vekris
12a726aee7 Support for Greek times and dates
Summary:
This adds support for greek times.

There are still some issues with expressions of the form:
```
9:30 - 11:00 την πέμπτη
```
Where `11:00 την πέμπτη` is parsed first (as 11:30 on Thu), instead of prioritizing `9:30 - 11:00` as the training data suggests. These tests are for the moment excluded from the corpus.

Reviewed By: patapizza

Differential Revision: D6376271

fbshipit-source-id: 2f31e058fb88386429070e3b51cd33f93b9c5936
2017-12-04 16:45:40 -08:00
Lukas Koebis
0d7939b246 Email/EN: Allow 'dot' in emails.
Summary: Duckling supports emails like "hey at things.com" but not "hey at things dot com". Let's add that.

Reviewed By: patapizza

Differential Revision: D6387471

fbshipit-source-id: 349a3a1ee55983daf5fda7579623f0c7bb32357f
2017-12-04 14:01:10 -08:00
Julien Odent
12000c2f86 Numeral/GA: fix old vigesimal + refactoring
Summary: Merging rules in two, fixing production rule.

Reviewed By: JonCoens, blandinw

Differential Revision: D6466793

fbshipit-source-id: 0fa143b049e9ccce7f4946e14bbd15fc37c324e1
2017-12-01 18:15:31 -08:00
Jana Šefčíková
7fe748ffec Numeral/GA: HashMap lookups for large regexes
Summary: Use hashMap oneToTenMap instead of guard.

Reviewed By: patapizza

Differential Revision: D6414352

fbshipit-source-id: a1788df0d3521a3a870e453294708d9fe2c10908
2017-12-01 10:15:24 -08:00
Michael Azzam
8fbd6a0414 Consolidate days of week and months
Summary: Refactor days of week and months rules to avoid mindless copy-pasta.

Reviewed By: patapizza

Differential Revision: D6438622

fbshipit-source-id: 489261b53ba50f6624996ad4581c2bf1206a8cc1
2017-11-29 17:00:37 -08:00
Julien Odent
d8c8533ed3 Time/EN: Fix remaining case sensitive rules
Summary: Fixes #112.

Reviewed By: blandinw

Differential Revision: D6408648

fbshipit-source-id: 944d8d19417b73d1e8a4996b5ea523f732f8dab7
2017-11-24 10:30:29 -08:00
Michal Nanasi
93e7010844 Backed out changeset 3b7abaab879c
Summary: Sigma is crashing in PCRe.

Reviewed By: codemiller, simonmar

Differential Revision: D6385023

fbshipit-source-id: cdd7e394606fb38df976d1bb81da414329ab7adf
2017-11-21 07:30:28 -08:00
Lukas Koebis
e361c9b694 Allow 'dot' in emails.
Summary: Duckling supports emails like "hey at things.com" but not "hey at things dot com". Let's add that.

Reviewed By: patapizza

Differential Revision: D6364483

fbshipit-source-id: 3b7abaab879c44c30c80c928de72c202068894a8
2017-11-20 12:15:29 -08:00
Julien Odent
54ccbf81df Numeral/DE: fix keiner + cleanup
Summary:
* fixed keine[rn]
* removed redundant `ruleInteger`
* replaced pattern matching with hashmap lookup
* renamed `dozensMap` to `tensMap`

Reviewed By: blandinw

Differential Revision: D6336252

fbshipit-source-id: 740734ab7b0b289adc4f466f966c4c5e59af75ad
2017-11-15 13:15:29 -08:00
igor-drozdov
29d776dee5 Added TimeGrain and Duration Dimensions to Russian language
Summary:
- Added Duration dimension to Russian language
- Added TimeGrain dimension to Russian language
- Refactored isNatural and isNaturalWith out of Duration helpers into Numeral helpers
- Implemented <integer> and a half rule for Russian Numeral
- Changed the type of inSeconds to polymorphic one
Closes https://github.com/facebook/duckling/pull/105

Reviewed By: blandinw

Differential Revision: D6312604

Pulled By: patapizza

fbshipit-source-id: 9ae237b4beb6915ff8da013230457937d8e56733
2017-11-15 10:45:24 -08:00