Commit Graph

268 Commits

Author SHA1 Message Date
Bill Cromie
197fef1b5e Ordinals with spaces, en-dashes and more!
Summary:
Adding more support for "natural" language compound ordinals:
"twenty nine", "twentynine", and "twenty–nine".

These tend to come up in text that has been transcribed from audio, informal user input, and more formal printed material respectively.
Closes https://github.com/facebook/duckling/pull/152

Reviewed By: panagosg7

Differential Revision: D6850514

Pulled By: patapizza

fbshipit-source-id: 1d3491655cfc1fb3faab0c3e196de1b2361ae95e
2018-02-01 14:30:44 -08:00
Julien Odent
b00fa512af AmountOfMoney: fixes
Summary:
* don't recursively compose cents
* don't allow decreasing ranges

Reviewed By: blandinw

Differential Revision: D6849132

fbshipit-source-id: ed6ca30388642c21e677a628971747a4fb3dfbef
2018-01-30 14:45:36 -08:00
Julien Odent
bef7a44fa8 Remove redundant brackets and language pragmas
Summary: .

Reviewed By: JonCoens

Differential Revision: D6838082

fbshipit-source-id: 94757bdb80c6d3c29a7a6554429940a1b7403108
2018-01-29 16:45:28 -08:00
Filipe Pereira
81a97575f6 PR for issue #148
Summary:
patapizza Did my best, but I'm neither fluent on Dutch or Haskell :) I tried keeping it consistent with the existing code, let me know if there's something I can improve!
Closes https://github.com/facebook/duckling/pull/149

Reviewed By: blandinw

Differential Revision: D6809767

Pulled By: patapizza

fbshipit-source-id: f92802b94ed7588a03219f63e74dbab475e950ab
2018-01-26 09:46:04 -08:00
Magnus Burton
7d864e38ab Added more synonyms to about <duration>
Summary:
Added some more synonyms to the duration part of the Swedish Duckling. This will match even more **valid** durations that match `about <duration>`.
Closes https://github.com/facebook/duckling/pull/145

Differential Revision: D6775860

Pulled By: patapizza

fbshipit-source-id: d0c844747e3bbb60c2500fb4b79aa53c2680ab48
2018-01-22 17:02:16 -08:00
vxovka
b964d02a7d Russian ordinals fixes and additions
Summary:
Fixed 3rd ordinal which didn't work for feminine and neuter genders in nominative.
Fixed when ordinals from 1 to 19 not recognized when used with capital letters.
Added support for ordinals in genitive for all genders.
Closes https://github.com/facebook/duckling/pull/141

Reviewed By: patapizza

Differential Revision: D6740190

Pulled By: panagosg7

fbshipit-source-id: a6a676d95890a7e156346f479c84a71c1b2bd17f
2018-01-21 23:00:41 -08:00
Julien Odent
7f21d1d7d7 Time/EN: Parse MM.DD.YYYY (or DD.MM.YYYY)
Summary:
Fixes #143.

Doesn't allow for MM.DD.YY (or DD.MM.YY) as this clashes with HHMMSS.

Reviewed By: blandinw

Differential Revision: D6765534

fbshipit-source-id: 1d1720e5bb292fdb3970e76475f35eab316048a8
2018-01-19 19:15:49 -08:00
Akshay Patil
d590f098dc Add type for Duration
Summary: Added missing `type` for Duration.

Reviewed By: patapizza

Differential Revision: D6717680

fbshipit-source-id: d1454758a3852bb5c06fb7a14224012cb3085a58
2018-01-16 15:15:27 -08:00
Julien Odent
da36ab8a80 Time: fix empty values for past time
Summary: "yesterday" would resolve to an entity without any `values`.

Reviewed By: JonCoens

Differential Revision: D6697432

fbshipit-source-id: 7b15727f92703842a2995210fdeb99c00be74bc3
2018-01-10 15:30:32 -08:00
Panagiotis Vekris
f28ad7a367 support for reference time param in web server
Summary:
PR for the issue #132
Closes https://github.com/facebook/duckling/pull/133

Reviewed By: patapizza

Differential Revision: D6661000

Pulled By: panagosg7

fbshipit-source-id: da061cb55e86c8747ff885df9881e270d22cdfa7
2018-01-10 12:45:37 -08:00
Julien Odent
ffdb976dd4 Time/SV: Add more examples
Summary: #140.

Reviewed By: panagosg7

Differential Revision: D6667848

fbshipit-source-id: 861fb87caf10cac64f0fe2eaa5186a70da6a3245
2018-01-05 12:15:40 -08:00
Filipe Pereira
9f78ebd801 PR for issue #136
Summary: Closes https://github.com/facebook/duckling/pull/139

Reviewed By: panagosg7

Differential Revision: D6661224

Pulled By: patapizza

fbshipit-source-id: 4802c2f66efd017577f3247e3b4abddcb7496c3f
2018-01-05 12:15:40 -08:00
Abdallatif Sulaiman
ce2f48fbb5 Quickfix for Arabic Numeral
Summary: Closes https://github.com/facebook/duckling/pull/137

Reviewed By: adelnobel

Differential Revision: D6660959

Pulled By: patapizza

fbshipit-source-id: 51169c63558b5a35f16bdaaada9649c88a4d2994
2018-01-05 11:45:40 -08:00
Mark Neumann
92e321e9ad Add note about -j1 flag for stack build
Summary:
I had trouble building the Docker image on a mac because `stack build` was trying to use all the cores and subsequently running out of memory. It took me a while to find the right flag, so I added a note to the dockerfile.
Closes https://github.com/facebook/duckling/pull/135

Differential Revision: D6655605

Pulled By: patapizza

fbshipit-source-id: 559c9fde175ad89f64b5c487afb77e3b7d5b982f
2018-01-03 06:00:44 -08:00
Abdallatif Sulaiman
89822776c6 Added Amount of Money Dimension to Arabic language
Summary: Closes https://github.com/facebook/duckling/pull/134

Reviewed By: JonCoens

Differential Revision: D6649920

Pulled By: patapizza

fbshipit-source-id: 9c647c84f5ae4f3dc26cb0c7aa74abb097ea001a
2018-01-02 14:00:51 -08:00
Jade Auer
fc574a448e add FB code of conduct
Summary:
Add FB-standard code of conduct for open source projects.
ref: https://fburl.com/aedakn3u
Closes https://github.com/facebook/duckling/pull/128

Reviewed By: patapizza

Differential Revision: D6629244

Pulled By: jda

fbshipit-source-id: 50ef889052f97757220157d68dc531f5e72bf86f
2018-01-02 08:15:29 -08:00
Abdallatif Sulaiman
2d726d3837 Added Quantity Dimension to Arabic language
Summary: Closes https://github.com/facebook/duckling/pull/127

Reviewed By: panagosg7

Differential Revision: D6616298

Pulled By: patapizza

fbshipit-source-id: dc774ff1e870bbd083a9cca8ee6f75db852afce9
2017-12-20 18:00:43 -08:00
Abdallatif Sulaiman
c056a0b46a Added Temperature Dimension to Arabic language
Summary: Closes https://github.com/facebook/duckling/pull/126

Reviewed By: panagosg7

Differential Revision: D6614995

Pulled By: patapizza

fbshipit-source-id: 35ff142a3fc8a498d9abbb80d28dc9be6cdcbc4d
2017-12-20 18:00:43 -08:00
Abdallatif Sulaiman
380457db8f Added Volume Dimension to Arabic language
Summary: Closes https://github.com/facebook/duckling/pull/125

Reviewed By: panagosg7

Differential Revision: D6614574

Pulled By: patapizza

fbshipit-source-id: 054ed04e0cc1cf79be31340ebb8b7fea3bc67f57
2017-12-20 18:00:43 -08:00
Abdallatif Sulaiman
1393098bcc Added Time Dimension to Arabic
Summary:
Hi, in this pr:
* Added time dimension to Arabic language, thanks to Hussein-Dahir & Yazeed-Obaid for writing time corpus.
* Fixed some bugs in numeral & ordinals and added more test cases for them.

Also, I don't really understand why do we use classifiers in time dimension?
Closes https://github.com/facebook/duckling/pull/123

Reviewed By: blandinw

Differential Revision: D6583313

Pulled By: patapizza

fbshipit-source-id: f7acdef0c032d7b7fd7d224832fdaf484d2df825
2017-12-19 14:30:42 -08:00
Newinfinite007
c133bad24a Hindi Language Numeral Dimension(minimalistic model). Tests passed.
Summary: Closes https://github.com/facebook/duckling/pull/119

Reviewed By: JonCoens

Differential Revision: D6597628

Pulled By: patapizza

fbshipit-source-id: 8bac0f686d6cecc38d9998e37042fe48f73530dc
2017-12-19 13:15:30 -08:00
Henri Dwyer
15b3988ec5 update MY pattern match to HashMap
Summary: Create hashmaps for the pattern matching rules, and make the regex matching read the value from the hashmap.

Reviewed By: patapizza

Differential Revision: D6592508

fbshipit-source-id: 0f4dd9bc548f71d0e6e52c938bd2e39bfd909c01
2017-12-18 13:00:35 -08:00
Henri Dwyer
10db72fd05 update KO pattern match to HashMap
Summary: Create hashmaps for the pattern matching rules, and make the regex matching read the value from the hashmap.

Reviewed By: panagosg7

Differential Revision: D6536790

fbshipit-source-id: ecb0bf28cf22b82ef10c3c2408eda15409189431
2017-12-15 11:45:28 -08:00
Julien Odent
fb3979d257 Numeral: move ruleFractions to common + cleanup numeral flag for Time
Summary:
* Closes PRs #115, #116.
* `Numeral`: Moves `ruleFractions` to common
* `Numeral/PT`: fixes dot in regex
* `Time`: Moves `isNumeralSafeToUse` to `isIntegerBetween` (more robust and cleaner)
* `Time/EN`: refactored `ruleCycleLastNextN` to `ruleDurationLastNext` (so it also accounts for non-safe Numerals, like "in a few days")

Reviewed By: blandinw

Differential Revision: D6502958

fbshipit-source-id: 6d9c08a23dab88f441aadade08d663c8e0da415d
2017-12-15 10:15:31 -08:00
Zahar Shimanchik
2274e40369 Added AmountOfMoney Dimension to Russian language
Summary: Closes https://github.com/facebook/duckling/pull/120

Reviewed By: patapizza

Differential Revision: D6520302

Pulled By: panagosg7

fbshipit-source-id: 2c9ac6e15ca3ee90b2bf5911ee62835966fcacd1
2017-12-12 12:15:30 -08:00
Julien Odent
6df3b26707 Numeral: common rule + supporting hindu-arabic numerals for Burmese
Summary:
* `ruleIntegerNumeric` was used in all languages but Burmese.
* it seems like the hindu-arabic numerals are slowly getting in Burmese (e.g. recent car plates)
* Moving the rule in `Duckling/Numeral/Common.hs`

Reviewed By: blandinw

Differential Revision: D6498349

fbshipit-source-id: e868dc9960f18f0781e4aa98a0dfcd14969537c9
2017-12-06 16:00:28 -08:00
Genki Furumi
56b57df153 HashMap lookups for large regexes
Summary: Replaced pattern matching with Hashmap.

Reviewed By: patapizza

Differential Revision: D6493427

fbshipit-source-id: 5ae9370387a738423724cabbe7c0d44c4889e185
2017-12-06 11:00:38 -08:00
Alex Torres
498e8b16e6 fix pt rules for numeral
Summary:
When I write "dois mil e duzentos" the result should be 2200, but duckling recognize the numbers separated and give the result:

`[{"dim":"number","body":"dois","value":{"value":2,"type":"value"},"start":0,"end":4},{"dim":"number","body":"mil","value":{"value":1000,"type":"value"},"start":5,"end":8},{"dim":"time","body":"mil","value":{"values":[],"value":"1000-01-01T00:00:00.000-07:53","grain":"year","type":"value"},"start":5,"end":8},{"dim":"number","body":"duzentos","value":{"value":200,"type":"value"},"start":11,"end":19}]`

Now with this commit, duckling gives the correct result:

`[{"dim":"number","body":"dois mil e duzentos","value":{"value":2200,"type":"value"},"start":0,"end":19}]`
Closes https://github.com/facebook/duckling/pull/117

Reviewed By: blandinw

Differential Revision: D6477925

Pulled By: patapizza

fbshipit-source-id: 26ab503cc8def739c51ceb5bae7546016ba65ad6
2017-12-04 18:00:36 -08:00
Panagiotis Vekris
12a726aee7 Support for Greek times and dates
Summary:
This adds support for greek times.

There are still some issues with expressions of the form:
```
9:30 - 11:00 την πέμπτη
```
Where `11:00 την πέμπτη` is parsed first (as 11:30 on Thu), instead of prioritizing `9:30 - 11:00` as the training data suggests. These tests are for the moment excluded from the corpus.

Reviewed By: patapizza

Differential Revision: D6376271

fbshipit-source-id: 2f31e058fb88386429070e3b51cd33f93b9c5936
2017-12-04 16:45:40 -08:00
Lukas Koebis
0d7939b246 Email/EN: Allow 'dot' in emails.
Summary: Duckling supports emails like "hey at things.com" but not "hey at things dot com". Let's add that.

Reviewed By: patapizza

Differential Revision: D6387471

fbshipit-source-id: 349a3a1ee55983daf5fda7579623f0c7bb32357f
2017-12-04 14:01:10 -08:00
Julien Odent
12000c2f86 Numeral/GA: fix old vigesimal + refactoring
Summary: Merging rules in two, fixing production rule.

Reviewed By: JonCoens, blandinw

Differential Revision: D6466793

fbshipit-source-id: 0fa143b049e9ccce7f4946e14bbd15fc37c324e1
2017-12-01 18:15:31 -08:00
Jana Šefčíková
7fe748ffec Numeral/GA: HashMap lookups for large regexes
Summary: Use hashMap oneToTenMap instead of guard.

Reviewed By: patapizza

Differential Revision: D6414352

fbshipit-source-id: a1788df0d3521a3a870e453294708d9fe2c10908
2017-12-01 10:15:24 -08:00
Michael Azzam
8fbd6a0414 Consolidate days of week and months
Summary: Refactor days of week and months rules to avoid mindless copy-pasta.

Reviewed By: patapizza

Differential Revision: D6438622

fbshipit-source-id: 489261b53ba50f6624996ad4581c2bf1206a8cc1
2017-11-29 17:00:37 -08:00
Julien Odent
d8c8533ed3 Time/EN: Fix remaining case sensitive rules
Summary: Fixes #112.

Reviewed By: blandinw

Differential Revision: D6408648

fbshipit-source-id: 944d8d19417b73d1e8a4996b5ea523f732f8dab7
2017-11-24 10:30:29 -08:00
Julien Odent
cc7cf74ff4 New locale instruction
Summary: to be more specific

Reviewed By: blandinw

Differential Revision: D6391421

fbshipit-source-id: 5c8e34b7a072ad2820a064c66a2364e866147630
2017-11-22 17:02:55 -08:00
Michal Nanasi
93e7010844 Backed out changeset 3b7abaab879c
Summary: Sigma is crashing in PCRe.

Reviewed By: codemiller, simonmar

Differential Revision: D6385023

fbshipit-source-id: cdd7e394606fb38df976d1bb81da414329ab7adf
2017-11-21 07:30:28 -08:00
Lukas Koebis
e361c9b694 Allow 'dot' in emails.
Summary: Duckling supports emails like "hey at things.com" but not "hey at things dot com". Let's add that.

Reviewed By: patapizza

Differential Revision: D6364483

fbshipit-source-id: 3b7abaab879c44c30c80c928de72c202068894a8
2017-11-20 12:15:29 -08:00
Julien Odent
54ccbf81df Numeral/DE: fix keiner + cleanup
Summary:
* fixed keine[rn]
* removed redundant `ruleInteger`
* replaced pattern matching with hashmap lookup
* renamed `dozensMap` to `tensMap`

Reviewed By: blandinw

Differential Revision: D6336252

fbshipit-source-id: 740734ab7b0b289adc4f466f966c4c5e59af75ad
2017-11-15 13:15:29 -08:00
igor-drozdov
29d776dee5 Added TimeGrain and Duration Dimensions to Russian language
Summary:
- Added Duration dimension to Russian language
- Added TimeGrain dimension to Russian language
- Refactored isNatural and isNaturalWith out of Duration helpers into Numeral helpers
- Implemented <integer> and a half rule for Russian Numeral
- Changed the type of inSeconds to polymorphic one
Closes https://github.com/facebook/duckling/pull/105

Reviewed By: blandinw

Differential Revision: D6312604

Pulled By: patapizza

fbshipit-source-id: 9ae237b4beb6915ff8da013230457937d8e56733
2017-11-15 10:45:24 -08:00
Igor Drozdov
f6492b5da0 Added Quantity dimension to Russian language
Summary: Closes https://github.com/facebook/duckling/pull/106

Reviewed By: blandinw

Differential Revision: D6312605

Pulled By: patapizza

fbshipit-source-id: 69ec673f95ec8a2d86ec207a6d75cd8ebfcdb4f6
2017-11-14 21:00:28 -08:00
Julien Odent
fb10a6e6ba Time/EN: Parse spelled out times + AM/PM
Summary:
When using speech recognition, we might see things like "six thirty six a m" or
"ten thirty p m".
Also fixed the argument order of `timeOfDayAMPM` to be more idiomatic.

Reviewed By: blandinw

Differential Revision: D6316542

fbshipit-source-id: 0008c049040219b3a1dd80d9e4661ba8a246fa7f
2017-11-14 13:30:26 -08:00
Julien Odent
98ff268918 Point to language-specific support in README
Summary: Highlights opportunities.

Reviewed By: JonCoens

Differential Revision: D6312814

fbshipit-source-id: dfa1f9a6bf6d83d7dfc69b88209a7242ad2f4aef
2017-11-13 10:00:37 -08:00
Julien Odent
277895193f Link to wit.ai in README
Summary:
Suggest to use wit.ai's built-in entities if running Duckling is not
an option.

Reviewed By: JonCoens

Differential Revision: D6312789

fbshipit-source-id: 08f1c5759cdd7761b456914cd5c42e1c111c6408
2017-11-13 10:00:37 -08:00
Julien Odent
436e4662d9 Time/NL: don't be too eager on days of week
Summary:
"d" would parse as "dinsdag" (Tuesday), "zo" would parse as "zondag" (Sunday, also means "so").
Made dots mandatory, to prevent further issues (e.g. "zon" means "sun").

Reviewed By: mullender

Differential Revision: D6312693

fbshipit-source-id: 58c5824e3ff174fc9c293c3f2d13e152c60e51de
2017-11-13 09:00:40 -08:00
Panagiotis Vekris
c6a7fedb7b Extract useful time-rule constructors to Time.Helpers
Summary: Some constructors for Time related rules are being reused among different language, so we can extract them in `Duckling/Time/Helpers.hs`.

Reviewed By: patapizza

Differential Revision: D6269106

fbshipit-source-id: 7d9969ce425ee27a2e1a32ea48932e16c7e6b1f1
2017-11-08 14:45:35 -08:00
Panagiotis Vekris
536f2844e3 Support Greek ordinals
Summary: Adding support for Greek ordinals

Reviewed By: patapizza

Differential Revision: D6263781

fbshipit-source-id: ff339ee51e4e8ad6b0c8f3fa75f5652391dbe48e
2017-11-08 11:00:31 -08:00
Igor Drozdov
5d2c5c78ba Added Distance and Volume Dimensions for Russian language
Summary:
- Added Distance Dimension for Russian language (RU)
- Added Volume Dimension for Russian language (RU)
- Extended `Duckling.Distance.Types.Unit` type definition by adding `Millimetre` representation
Closes https://github.com/facebook/duckling/pull/101

Reviewed By: JonCoens

Differential Revision: D6254070

Pulled By: patapizza

fbshipit-source-id: 579f7a259f76ff1c23ccfe2371afea385eb56aa1
2017-11-08 11:00:31 -08:00
Abdallatif Sulaiman
8cbdabef09 More rules to AR Ordinal dimension
Summary:
- Add more tests to Ar ordinal corpus
- Add HashMap lookups and add more rules to composite numbers
Closes https://github.com/facebook/duckling/pull/100

Reviewed By: JonCoens

Differential Revision: D6254264

Pulled By: patapizza

fbshipit-source-id: 6d50286f6bd3a60e34b0eb855fb38e044c3c39dc
2017-11-08 10:45:29 -08:00
Julien Odent
87c875f1e5 Fix Travis links in README
Summary: We've graduated from `facebookincubator` to `facebook`.

Reviewed By: JonCoens

Differential Revision: D6272667

fbshipit-source-id: d433c8f1417d94f703fc455a466be64253ed85b8
2017-11-08 10:00:36 -08:00
Julien Odent
bc3cd146cd fbshipit-source-id: f5a450fa3c872707fffeb9c241db77e44acd0f26 2017-11-08 09:33:19 -08:00