Summary:
In theory, years can be negative. In practice, this is rarely seen and when it is, it's usually with the B.C. postfix.
Updating `ruleYearLatent` to not accept negative years for all languages.
Reviewed By: codyohl
Differential Revision: D33895871
fbshipit-source-id: 818000104da825aab91be7fa2a72704aa350a91a
Summary:
Improves the recognition of German time approximation language and removes a single error in the rule of <time-of-day> approximately.
Pull Request resolved: https://github.com/facebook/duckling/pull/435
Reviewed By: patapizza
Differential Revision: D24934281
Pulled By: chessai
fbshipit-source-id: 641bcb6a7e5c26e66c735fe13bccae9b7a8909ae
Summary:
"so" is an adverb in German: https://github.com/wit-ai/wit/issues/1860
It's also a short form for "Sonntag" (Sunday); making the dot mandatory.
Reviewed By: haoxuany
Differential Revision: D22900791
fbshipit-source-id: 8dc873f79a21ca2add074f9c664e84fae56f1e67
Summary: Added German langauage support for all computed holidays currently supported in English. Also created HolidayHelpers, which contains more complex holiday calculations used across languages.
Reviewed By: patapizza
Differential Revision: D17386163
fbshipit-source-id: 9dd88f8b0d699e5d7254a5ba7114bfa01b15176a
Summary:
Add 'the day before yesterday' rule for German, and a test for it. Also another test
Pull Request resolved: https://github.com/facebook/duckling/pull/324
Reviewed By: patapizza
Differential Revision: D13931961
Pulled By: haoxuany
fbshipit-source-id: 11d9ebd879be3e4d28a762d5638ee300a4132445
Summary:
- took regex from EN (which is working)
- added test cases from 2016-01-01 to 2018-12-31
- change non-breaking space to regular space
Pull Request resolved: https://github.com/facebook/duckling/pull/282
Reviewed By: chinmay87
Differential Revision: D13551882
Pulled By: patapizza
fbshipit-source-id: e254be1beaed21226741c55f243ac21eaa06d636
Summary:
Return the canonical holiday name if the input had a holiday in it and the date resolves to the date of the holiday.
Add `holiday` to `TimeData`.
Set it in the holiday rule.
Make sure it propogates appropriately in `predNth` and `intersect`.
Make sure it doesn't propogate in other rules.
Add `holiday` to `TimeValue` and set it during `resolve`
Reviewed By: patapizza
Differential Revision: D7441387
fbshipit-source-id: d57602fd294dc6149ab044c05065c1b9250cb331
Summary: Support lowercase time zones
Reviewed By: patapizza
Differential Revision: D7114137
fbshipit-source-id: 906c4fa5305aea8990e8a5d480ceeec4b584d7db
Summary:
When using speech recognition, we might see things like "six thirty six a m" or
"ten thirty p m".
Also fixed the argument order of `timeOfDayAMPM` to be more idiomatic.
Reviewed By: blandinw
Differential Revision: D6316542
fbshipit-source-id: 0008c049040219b3a1dd80d9e4661ba8a246fa7f
Summary: Some constructors for Time related rules are being reused among different language, so we can extract them in `Duckling/Time/Helpers.hs`.
Reviewed By: patapizza
Differential Revision: D6269106
fbshipit-source-id: 7d9969ce425ee27a2e1a32ea48932e16c7e6b1f1
Summary:
- Fixes#89. "<number> <number>" would parse as Time if in the right range.
- Applied same rule for all languages. Note that for Italian and Polish, I updated "<hour> <minute>" tests to be in the form "at <hour> <minute>".
- Replaced `liftM2` with more generic `and|or . sequence [f1, f2, ...]`.
Reviewed By: blandinw
Differential Revision: D5992879
fbshipit-source-id: 5409ffb
Summary:
We noticed that using UTF-8 characters directly in regexes work.
Hence converting back the escaped characters for readability and maintenance.
Reviewed By: blandinw
Differential Revision: D5787146
fbshipit-source-id: e5a4b9a
Summary:
Combined each of seasons, instants, and holidays into a data list and a
function to generate the list of Rules.
*Instants = today, tomorrow, now, end of year, etc.
Reviewed By: patapizza
Differential Revision: D5730896
fbshipit-source-id: 23170e7
Summary:
The German ruleset has 12 separate rules for months, and 7 for days. This
change replaces those with a list of months/days and a single function
to create a list of rules from those. This is the same approach as is currently in the English ruleset.
Reviewed By: patapizza
Differential Revision: D5728656
fbshipit-source-id: 8590f4a
Summary: This is a simple refactor that uses the weekend helper for all languages
Reviewed By: patapizza
Differential Revision: D5677330
fbshipit-source-id: 9984539
Summary:
Fixes#65.
* fixes US holidays
* Black Friday is actually the first day after Thanksgiving day (not necessary the fourth Friday of November)
Reviewed By: JonCoens
Differential Revision: D5533906
fbshipit-source-id: 1824cba
Summary:
This PR contains various smaller but - at least on my data - important performance improvements for matching of German time and time range expressions.
I evaluated this on approx 11.000 time and time range expressions taken from emails (rather formal business travel requests) that have been manually annotated with the "true" time. Comparing this branch to the current master (`d6f8dd`) I get e.g. approx. 80% of the duckling results within +/- 1h of the true value (hours are the smallest grain in my data), vs. only 70% in the master. Other indicators I checked (time/range confusion, other thresholds, failures to find anything in the first place, etc.) were all improved as well.
**Changes**:
* [significant performance plus] added a rule `ruleDateDateInterval` that handles variations of "13.-15.10." correctly. Here the common case is that "13." refers to "13.10." and not "13.CURRENTMONTH". I didn't see an obvious way to fix that in the `<datetime> - <datetime>` rule.
* [significant performance plus] In `ruleMmdd` (which matches expressions like "13.03." in German), I made the last dot optional. At least in less formal text this is quite common to be forgotten. Also here and in `ruleDateDateInterval` I changed the order of the terms in the regular expression matching the month to prefer matching e.g. "10" over matching "1"+"no dot".
* [minor] treat "14/15Uhr" the same as "14-15Uhr"
* [minor] Extended "bis" to also match "bis zum" and "auf den" (e.g. in "von Montag bis zum Freitag" or "von Dienstag auf Mittwoch")
* [minor] Changed `hh:mm` matching to also get the rather esoteric expression "17h00" - should do no harm.
Closes https://github.com/facebookincubator/duckling/pull/54
Reviewed By: blandinw
Differential Revision: D5301815
Pulled By: patapizza
fbshipit-source-id: 8766caf
Summary:
* In DE `frühestens` and `spätestens` act implicitly as `nach` and `vor` (after and before) on times and may also appear after the time
* The rule `ruleTimeofdayTimeofdayInterval` does match `9Uhr-10` but not the
way more common expression `9-10Uhr`; added the same rule with the
second time as non-latent; actually I am not sure whether the original
rule makes sense at all
* Simple extension of `intersect by ,` to THE formal way in DE to express
a date (i.e. `Freitag, der 13.03.2013`)
General remark: I used UTF-8 characters albeit I saw that the other rules and examples use escaped hex encoding for e.g. German umlaute. If there is any reason to do that (it is not very readable), I will of course change that.
Closes https://github.com/facebookincubator/duckling/pull/19
Reviewed By: niteria
Differential Revision: D5070052
Pulled By: patapizza
fbshipit-source-id: 990ad08
Summary:
This is analogous to
[Duckling] Don't produce trivially empty Tokens
but that change did that for intersect, this one
deals with interval.
Reviewed By: patapizza
Differential Revision: D5039215
fbshipit-source-id: 95bd821
Summary:
`intersectMB` was a name used for the purpose of migrating.
This is the last part of the migration.
Reviewed By: patapizza
Differential Revision: D4906098
fbshipit-source-id: a70af78
Summary:
This continues the work from:
"[Duckling] Don't produce trivially empty Tokens"
All the Rules should use intervalMB from now on.
Reviewed By: patapizza
Differential Revision: D4906072
fbshipit-source-id: 277b961
Summary:
We can detect certain kinds of contradictions sooner,
producing a token with an unresolvable Predicate is wasteful.
For a text like:
```
"Demain apres midi 14h 15 h 16h vendredi 14 a 15h"
```
it could produce 7000 tokens with empty predicates.
After this change it produces none and we get a 4x improvement in
time and 6x improvement in allocations.
Note I only covered `ruleIntersect*` here. I need to do this for
other instances as well.
Reviewed By: JonCoens
Differential Revision: D4871078
fbshipit-source-id: 9f0e7ad
Summary:
This is a very common pattern (>1k occurrences).
Replacing it with something shorter makes the rules a bit less
boilerplate-y.
Feel free to bikeshed the name, I can easily redo the codemod.
Reviewed By: patapizza
Differential Revision: D4848864
fbshipit-source-id: 7baeee3
Summary:
This makes the code easier to read.
I'm not attached to naming, but this is
standard terminology from topology.
Reviewed By: JonCoens, patapizza
Differential Revision: D4848740
fbshipit-source-id: 79c2c20
