Summary:
Fixes#75.
I created a list of regex expressions and their corresponding units. I then mapped over them to create two rules for each expression: one for numeral quantities and one for single (a/an) quantities.
Reviewed By: patapizza
Differential Revision: D5532950
fbshipit-source-id: 63e35bd
Summary:
Fixes#65.
* fixes US holidays
* Black Friday is actually the first day after Thanksgiving day (not necessary the fourth Friday of November)
Reviewed By: JonCoens
Differential Revision: D5533906
fbshipit-source-id: 1824cba
Summary:
In French, the form "at hh" is not valid (it requires an hour indicator).
This fixes false positives such as in "John a un rendez-vous."
Fixes https://github.com/wit-ai/wit/issues/666.
Reviewed By: JonCoens
Differential Revision: D5530713
fbshipit-source-id: ecee1e5
Summary:
* `Duration` before/after `Time` now resolves with the lowest grain
* "now" has an undefined grain `NoGrain`, as depending on the context it might mean different things, as opposed to "right now"
Before:
`day after tomorrow` -> `day` grain
`1 day after tomorrow` -> `hour` grain
Given that the reference date/time is `2013-02-12T04:30:00`.
`one year from now` -> `2014-02-01T00:00:00` with `month` grain.
`one year from today` -> `2014-02-01T00:00:00` with `month` grain.
After:
`day after tomorrow` -> `day` grain
`1 day after tomorrow` -> `day` grain
`one year from now` -> `2014-02-12T04:30:00` with `month` grain (remains the same).
`one year from today` -> `2014-02-12T00:00:00` with `day` grain.
For other `Time` entities involving `Duration`, such as "in + `Duration`", the behavior remains the same: shift to the lower grain (the intent is not precise).
Reviewed By: l5t, blandinw
Differential Revision: D5467164
fbshipit-source-id: b63b6a4
Summary:
Repeating version bounds on executables that depend on
`duckling` library is needless bureaucracy.
Reviewed By: JonCoens
Differential Revision: D5478388
fbshipit-source-id: e18f1b8
Summary:
MM/YY is a common format for dates in India,UK and other parts of the world.Have added testcases in `Time/EN/corpus.hs` ,however it conflicts with one of the original(2/15 is output now as Feb. 2015 and not the 15th of February).
Closes https://github.com/facebookincubator/duckling/pull/59
Reviewed By: niteria
Differential Revision: D5455881
Pulled By: patapizza
fbshipit-source-id: 23b73a5
Summary:
Today things like `at single`, `at a few`, `at a couple of` would return a `Time`.
Discussed with blandinw to do this very explicit hack right now until other use cases show up.
Reviewed By: niteria
Differential Revision: D5325369
fbshipit-source-id: aec0402
Summary:
The one restriction on using DuplicateRecordFields is that record
selectors have to be imported under their constructor, instead of as
top-level functions. Do this for si_sigma so D5242707 passes the compat
check.
Reviewed By: watashi
Differential Revision: D5326634
fbshipit-source-id: 74ec0dd
Summary:
I fixed some bugs I found in Portuguese. This is my first attempt to contribute so let me know if there's any thing I could do better next time! thanks! awesome project!
Closes https://github.com/facebookincubator/duckling/pull/56
Differential Revision: D5318968
Pulled By: patapizza
fbshipit-source-id: 94ff30f
Summary:
This PR contains various smaller but - at least on my data - important performance improvements for matching of German time and time range expressions.
I evaluated this on approx 11.000 time and time range expressions taken from emails (rather formal business travel requests) that have been manually annotated with the "true" time. Comparing this branch to the current master (`d6f8dd`) I get e.g. approx. 80% of the duckling results within +/- 1h of the true value (hours are the smallest grain in my data), vs. only 70% in the master. Other indicators I checked (time/range confusion, other thresholds, failures to find anything in the first place, etc.) were all improved as well.
**Changes**:
* [significant performance plus] added a rule `ruleDateDateInterval` that handles variations of "13.-15.10." correctly. Here the common case is that "13." refers to "13.10." and not "13.CURRENTMONTH". I didn't see an obvious way to fix that in the `<datetime> - <datetime>` rule.
* [significant performance plus] In `ruleMmdd` (which matches expressions like "13.03." in German), I made the last dot optional. At least in less formal text this is quite common to be forgotten. Also here and in `ruleDateDateInterval` I changed the order of the terms in the regular expression matching the month to prefer matching e.g. "10" over matching "1"+"no dot".
* [minor] treat "14/15Uhr" the same as "14-15Uhr"
* [minor] Extended "bis" to also match "bis zum" and "auf den" (e.g. in "von Montag bis zum Freitag" or "von Dienstag auf Mittwoch")
* [minor] Changed `hh:mm` matching to also get the rather esoteric expression "17h00" - should do no harm.
Closes https://github.com/facebookincubator/duckling/pull/54
Reviewed By: blandinw
Differential Revision: D5301815
Pulled By: patapizza
fbshipit-source-id: 8766caf
Summary:
This adds two new targets:
* `:duckling-expensive` - meant to have inputs for which running Duckling is expensive
* `:duckling-request-sample` - meant to have a random sample of inputs
The reason to have 2 of them is that they measure different things.
`:duckling-expensive` is correlated with failures,
`:duckling-request-sample` is correlated with cost.
I intend to add basic instruction on how to use them for
benchmarking/profiling soon.
Reviewed By: patapizza
Differential Revision: D5301554
fbshipit-source-id: f73fd85
Summary:
We build `duckling` again to test the source distribution.
It's a low-signal check that's too costly at the moment.
Reviewed By: patapizza
Differential Revision: D5301108
fbshipit-source-id: cc651a4
Summary:
e.g. "New York from 10-6 to 10-22" currently extracts: HH-MM. Instead, it should extract mm-dd i.e. October 10th to October 22nd.
Closes https://github.com/facebookincubator/duckling/pull/48
Reviewed By: niteria
Differential Revision: D5292473
Pulled By: patapizza
fbshipit-source-id: 04f1a4b
Summary:
'one' is a latent time of day.
Restricting a couple of rules to accept non-latent time tokens.
Reviewed By: blandinw
Differential Revision: D5293972
fbshipit-source-id: 07cdb9b
Summary:
Transform large case matches into HashMap lookups.
Add an extra example for a rule set that wasn't tested before.
Reviewed By: patapizza
Differential Revision: D5253349
fbshipit-source-id: 303dbca
Summary:
not sure about this. Maybe I need some guidance.
Closes https://github.com/facebookincubator/duckling/pull/42
Reviewed By: blandinw
Differential Revision: D5228520
Pulled By: patapizza
fbshipit-source-id: 4f99cc5
Summary:
The existing "mm/dd" rules only accepts format like "05/27"; However, in practice there might be extra spaces like "05 / 27", "05/ 27". The pull requests tweaks the regex to accept extra space.
Closes https://github.com/facebookincubator/duckling/pull/31
Reviewed By: niteria
Differential Revision: D5147118
Pulled By: patapizza
fbshipit-source-id: f6a5069
Summary:
Add the [Cantonese](https://en.wikipedia.org/wiki/Cantonese) (the official spoken language used in Hong Kong) support to date time
- updated Duration ZH corpus
- updated Time ZH rules and corpus
- updated TimeGrain ZH rules
Closes https://github.com/facebookincubator/duckling/pull/24
Reviewed By: patapizza
Differential Revision: D5143947
Pulled By: niteria
fbshipit-source-id: 9107d05
Summary:
* In DE `frühestens` and `spätestens` act implicitly as `nach` and `vor` (after and before) on times and may also appear after the time
* The rule `ruleTimeofdayTimeofdayInterval` does match `9Uhr-10` but not the
way more common expression `9-10Uhr`; added the same rule with the
second time as non-latent; actually I am not sure whether the original
rule makes sense at all
* Simple extension of `intersect by ,` to THE formal way in DE to express
a date (i.e. `Freitag, der 13.03.2013`)
General remark: I used UTF-8 characters albeit I saw that the other rules and examples use escaped hex encoding for e.g. German umlaute. If there is any reason to do that (it is not very readable), I will of course change that.
Closes https://github.com/facebookincubator/duckling/pull/19
Reviewed By: niteria
Differential Revision: D5070052
Pulled By: patapizza
fbshipit-source-id: 990ad08
Summary:
The numerical ordinal matching rule in DE is too broad. An ordinal like "1." may not be proceeded or followed by numbers.
* Added negative lookbehind - avoids matching the first "1." in "1.1" as an ordinal.
* Added negative lookahead - avoids matching the second "1." in "1.1. as an ordinal
Closes https://github.com/facebookincubator/duckling/pull/18
Reviewed By: patapizza
Differential Revision: D5069200
Pulled By: niteria
fbshipit-source-id: 0583076
