Summary:
Return the canonical holiday name if the input had a holiday in it and the date resolves to the date of the holiday.
Add `holiday` to `TimeData`.
Set it in the holiday rule.
Make sure it propogates appropriately in `predNth` and `intersect`.
Make sure it doesn't propogate in other rules.
Add `holiday` to `TimeValue` and set it during `resolve`
Reviewed By: patapizza
Differential Revision: D7441387
fbshipit-source-id: d57602fd294dc6149ab044c05065c1b9250cb331
Summary:
* Added periodic holidays for all English locales
* Fixed father's day in Australia and New Zealand
Reviewed By: chinmay87
Differential Revision: D7450426
fbshipit-source-id: 9ddcf7f08e73e2bb501541d9697d64b7e99a8ef1
Summary: remove `ruleLastTuesdayLastJuly`, it does the exact same thing as `ruleLastTime`
Reviewed By: patapizza
Differential Revision: D7445836
fbshipit-source-id: 95667db4ce376223acce0ea36a6a0493946ceea1
Summary: Add EN support for last night and late last night
Reviewed By: patapizza
Differential Revision: D7369829
fbshipit-source-id: bd1dbecf247e428307adfae3e8e83840f6de77b1
Summary: Added rules for the beginning / end of year / month / week
Reviewed By: patapizza
Differential Revision: D7356994
fbshipit-source-id: 43489b3709b8ab586384321cf9bc79c45668162a
Summary:
Hello,
I wanted to fix problem with two months - May and November for PL language, since it is what I need to use in [wit.ai](http://wit.ai), but.. I'm not sure how to do that. I've created two failing tests, checked `Rules`, but there are already some rules which are responsible for that cases:
`20 listopada` :
`, ( "November" , "listopad|listopada|listopadowi|listopadem|listopadzie|lis\\.?|list\\.?" )`
`20 maja`:
`, ( "May" , "maj|maja|majowi|majem|maju" )`
Can you help me somehow fix that issues because I'm not sure how I can force Duckling to use proper regexp? :)
Closes https://github.com/facebook/duckling/pull/161
Reviewed By: patapizza
Differential Revision: D7353495
Pulled By: panagosg7
fbshipit-source-id: b0c70172e4cc9dbd4635ed64e89ffb463b643f90
Summary:
"DD MM YYYY", "MM DD YYYY" parsed only as a phone number, but it is also a popular datetime format.
I have added new time patterns and tests for this cases.
Closes https://github.com/facebook/duckling/pull/163
Reviewed By: patapizza
Differential Revision: D7320048
Pulled By: panagosg7
fbshipit-source-id: 5ccf1ccca173126f5d2b9b8b2664181f3b13f78b
Summary: Add an option to return latent time entities. This can be used when one is pretty certain that the input contains a datetime.
Reviewed By: patapizza
Differential Revision: D7254245
fbshipit-source-id: e9e0503cace2691804056fcebdc18fd9090fb181
Summary:
* added periodic holidays for all existing English locales
* moved Independence Day from common to US, as it is country-dependent
* moved Memorial Day from common to US, as it is country-dependent (fixed the one for Canada)
* added Decoration Day as an alias for US Memorial Day
Reviewed By: chinmay87
Differential Revision: D7294129
fbshipit-source-id: 89cdbf91b4c43e1996fa1f4509eac1d1b1978197
Summary: Support lowercase time zones
Reviewed By: patapizza
Differential Revision: D7114137
fbshipit-source-id: 906c4fa5305aea8990e8a5d480ceeec4b584d7db
Summary:
patapizza Did my best, but I'm neither fluent on Dutch or Haskell :) I tried keeping it consistent with the existing code, let me know if there's something I can improve!
Closes https://github.com/facebook/duckling/pull/149
Reviewed By: blandinw
Differential Revision: D6809767
Pulled By: patapizza
fbshipit-source-id: f92802b94ed7588a03219f63e74dbab475e950ab
Summary: "yesterday" would resolve to an entity without any `values`.
Reviewed By: JonCoens
Differential Revision: D6697432
fbshipit-source-id: 7b15727f92703842a2995210fdeb99c00be74bc3
Summary:
Hi, in this pr:
* Added time dimension to Arabic language, thanks to Hussein-Dahir & Yazeed-Obaid for writing time corpus.
* Fixed some bugs in numeral & ordinals and added more test cases for them.
Also, I don't really understand why do we use classifiers in time dimension?
Closes https://github.com/facebook/duckling/pull/123
Reviewed By: blandinw
Differential Revision: D6583313
Pulled By: patapizza
fbshipit-source-id: f7acdef0c032d7b7fd7d224832fdaf484d2df825
Summary:
* Closes PRs #115, #116.
* `Numeral`: Moves `ruleFractions` to common
* `Numeral/PT`: fixes dot in regex
* `Time`: Moves `isNumeralSafeToUse` to `isIntegerBetween` (more robust and cleaner)
* `Time/EN`: refactored `ruleCycleLastNextN` to `ruleDurationLastNext` (so it also accounts for non-safe Numerals, like "in a few days")
Reviewed By: blandinw
Differential Revision: D6502958
fbshipit-source-id: 6d9c08a23dab88f441aadade08d663c8e0da415d
Summary:
This adds support for greek times.
There are still some issues with expressions of the form:
```
9:30 - 11:00 την πέμπτη
```
Where `11:00 την πέμπτη` is parsed first (as 11:30 on Thu), instead of prioritizing `9:30 - 11:00` as the training data suggests. These tests are for the moment excluded from the corpus.
Reviewed By: patapizza
Differential Revision: D6376271
fbshipit-source-id: 2f31e058fb88386429070e3b51cd33f93b9c5936
Summary: Refactor days of week and months rules to avoid mindless copy-pasta.
Reviewed By: patapizza
Differential Revision: D6438622
fbshipit-source-id: 489261b53ba50f6624996ad4581c2bf1206a8cc1
Summary:
When using speech recognition, we might see things like "six thirty six a m" or
"ten thirty p m".
Also fixed the argument order of `timeOfDayAMPM` to be more idiomatic.
Reviewed By: blandinw
Differential Revision: D6316542
fbshipit-source-id: 0008c049040219b3a1dd80d9e4661ba8a246fa7f
Summary:
"d" would parse as "dinsdag" (Tuesday), "zo" would parse as "zondag" (Sunday, also means "so").
Made dots mandatory, to prevent further issues (e.g. "zon" means "sun").
Reviewed By: mullender
Differential Revision: D6312693
fbshipit-source-id: 58c5824e3ff174fc9c293c3f2d13e152c60e51de
Summary: Some constructors for Time related rules are being reused among different language, so we can extract them in `Duckling/Time/Helpers.hs`.
Reviewed By: patapizza
Differential Revision: D6269106
fbshipit-source-id: 7d9969ce425ee27a2e1a32ea48932e16c7e6b1f1
Summary:
* add flag for this/next/last time
* fix thanskgiving in EN locales
* `analyzedRangeTest` helper with `rangeTests` for `Time/EN`
Reviewed By: blandinw
Differential Revision: D6191209
fbshipit-source-id: 6eaa117
Summary: The date format changes between EN locales (https://en.wikipedia.org/wiki/Date_format_by_country). This diff fixes how dates are handled in each locale.
Reviewed By: patapizza
Differential Revision: D6156147
fbshipit-source-id: 22f296c
Summary:
One of the most common ways of saying "last \<day-of-the-week\>" in Swedish is saying "i \<day-of-the-week\>s", for example "i tisdags" (last Tuesday) or "i lördags" (last Saturday). This pull request adds support for this.
Closes https://github.com/facebookincubator/duckling/pull/92
Reviewed By: blandinw, kodafb
Differential Revision: D6064814
Pulled By: patapizza
fbshipit-source-id: 6ea5466
Summary:
`ter` also means `to have`, which is very common.
`ter` is a very rare form for `Tuesday`, only used in calendar-like contexts
Reviewed By: blandinw
Differential Revision: D6066506
fbshipit-source-id: c8cd231
Summary:
* `en_CA` locale
* In Canada, Thanksgiving Day is the second Monday of October.
* Black Friday is the same as the US.
* However Canada observes both DDMM and MMDD formats. Defer to later, falling back to US.
Reviewed By: blandinw
Differential Revision: D6058909
fbshipit-source-id: 3d4e05e
Summary:
* Moving `ruleNationalDay` from `ZH` rules to specific locales: `zh_CN`, `zh_HK`, `zh_MO`
* Fixed National Day for `zh_TW`.
Reviewed By: blandinw
Differential Revision: D6057565
fbshipit-source-id: 8f9f2ab
Summary:
* Locales support for the library, following `<Lang>_<Region>` with ISO 639-1 code for `<Lang>` and ISO 3166-1 alpha-2 code for `<Region>` (#33)
* `Locale` opaque type (composite of `Lang` and `Region`) with `makeLocale` smart constructor to only allow valid `(Lang, Region)` combinations
* API: `Context`'s `lang` parameter has been replaced by `locale`, with optional `Region` and backward compatibility.
* `Rules/<Lang>.hs` exposes
- `langRules`: cross-locale rules for `<Lang>`, from `<Dimension>/<Lang>/Rules.hs`
- `localeRules`: locale-specific rules, from `<Dimension>/<Lang>/<Region>/Rules.hs`
- `defaultRules`: `langRules` + specific rules from select locales to ensure backward-compatibility
* Corpus, tests & classifiers
- 1 classifier per locale, with default classifier (`<Lang>_XX`) when no locale provided (backward-compatible)
- Default classifiers are built on existing corpus
- Locale classifiers are built on
- `<Dimension>/<Lang>/Corpus.hs` exposes a common `corpus` to all locales of `<Lang>`
- `<Dimension>/<Lang>/<Region>/Corpus.hs` exposes `allExamples`: a list of examples specific to the locale (following `<Dimension>/<Lang>/<Region>/Rules.hs`).
- Locale classifiers use the language corpus extended with the locale examples as training set.
- Locale examples need to use the same `Context` (i.e. reference time) as the language corpus.
- For backward compatibility, `<Dimension>/<Lang>/Corpus.hs` can expose also `defaultCorpus`, which is `corpus` augmented with specific examples. This is controlled by `getDefaultCorpusForLang` in `Duckling.Ranking.Generate`.
- Tests run against each classifier to make sure runtime works as expected.
* MM/DD (en_US) vs DD/MM (en_GB) example to illustrate
Reviewed By: JonCoens, blandinw
Differential Revision: D6038096
fbshipit-source-id: f29c28d
Summary:
- Fixes#89. "<number> <number>" would parse as Time if in the right range.
- Applied same rule for all languages. Note that for Italian and Polish, I updated "<hour> <minute>" tests to be in the form "at <hour> <minute>".
- Replaced `liftM2` with more generic `and|or . sequence [f1, f2, ...]`.
Reviewed By: blandinw
Differential Revision: D5992879
fbshipit-source-id: 5409ffb
Summary: Consolidate HE language named-months in the same style of other language, particularly EN. Translations were taken from previous namedMonth# rules and merged into the combined rule. Hebrew-speaking verification would be helpful to verify the integrity of the translations, although they were already taken from the existing code.
Reviewed By: blandinw
Differential Revision: D5839625
fbshipit-source-id: d53f9c7
Summary: Changed the duckling Polish days and months rules to be in line with the new style including all in one rules
Reviewed By: blandinw
Differential Revision: D5903171
fbshipit-source-id: 7f2ba60
Summary: In Romanian, `sa` is fairly common: hai sa ne vedem (let's see), hai sa mergem (let's go).
Reviewed By: blandinw
Differential Revision: D5801345
fbshipit-source-id: db677e4
Summary:
We noticed that using UTF-8 characters directly in regexes work.
Hence converting back the escaped characters for readability and maintenance.
Reviewed By: blandinw
Differential Revision: D5787146
fbshipit-source-id: e5a4b9a
Summary:
Remove hardcoded named-days and named-months, and
replace them with ruleDaysOfWeek and ruleMonths.
Reviewed By: patapizza
Differential Revision: D5742209
fbshipit-source-id: 339fc0a
Summary:
The Galeic ruleset has 12 separate rules for months, and 7 for days. This
change replaces those with a list of months/days and a single function
to create a list of rules from those. This is the same approach as is currently in the English ruleset.
Reviewed By: patapizza
Differential Revision: D5756222
fbshipit-source-id: ac4bc42
Summary:
Combined each of seasons, instants, and holidays into a data list and a
function to generate the list of Rules.
*Instants = today, tomorrow, now, end of year, etc.
Reviewed By: patapizza
Differential Revision: D5730896
fbshipit-source-id: 23170e7
Summary:
The German ruleset has 12 separate rules for months, and 7 for days. This
change replaces those with a list of months/days and a single function
to create a list of rules from those. This is the same approach as is currently in the English ruleset.
Reviewed By: patapizza
Differential Revision: D5728656
fbshipit-source-id: 8590f4a
Summary: Consolidated all the days of week rules into one rule, and did the same for all the month rules.
Reviewed By: patapizza
Differential Revision: D5721202
fbshipit-source-id: 2b4a56f
Summary: Consolidated the previous days of weeks and month names in french duckling file to become only 2 rules. Allows for more concise, updated code.
Reviewed By: patapizza
Differential Revision: D5710056
fbshipit-source-id: 816ef88
Summary: Changed Danish time rules to use ruleDaysOfWeek and ruleMonths.
Reviewed By: patapizza
Differential Revision: D5709782
fbshipit-source-id: aa03065
Summary:
There are problems in the ordinal recognition for Swedish. The most severe one is that all the numbers above 15 are actually Danish, not Swedish. Apart from that digits and digits followed by a dot are considered ordinals.
This pull request fixes this and also adds support for ordinals up to 100. The structure of the code is similar to in the ordinal recognition in English. Tests are also updated, both the ordinal tests and the time tests where incorrect ordinals were used.
Closes https://github.com/facebookincubator/duckling/pull/86
Reviewed By: JonCoens
Differential Revision: D5698145
Pulled By: patapizza
fbshipit-source-id: c31d7bc
Summary: In Portuguese, "um" means the numeral "one" and the article "a".
Reviewed By: bfiss
Differential Revision: D5703396
fbshipit-source-id: 92ed04f
Summary:
Remove hardcoded named-days and named-months, and
replace them with ruleDaysOfWeek and ruleMonths.
Reviewed By: patapizza
Differential Revision: D5695475
fbshipit-source-id: d30557f
Summary: Consolidating the rules for months and days of the week in Italian following the pattern seen in English.
Reviewed By: patapizza
Differential Revision: D5665259
fbshipit-source-id: 45d6c3b
Summary: This is a simple refactor that uses the weekend helper for all languages
Reviewed By: patapizza
Differential Revision: D5677330
fbshipit-source-id: 9984539
Summary:
We don't allow matches adjacent to a character of the same class.
We were treating uppercase and lowercase characters differently.
"jon Friday" wouldn't match "on" but "Jon Friday" would.
Reviewed By: blandinw
Differential Revision: D5653681
fbshipit-source-id: be67358
Summary: Added ruleIntervalDDDDMonth to EN to handle cases such as "23rd to 26th Oct" and "1-8 september"
Reviewed By: patapizza
Differential Revision: D5637280
fbshipit-source-id: a1fdcd2
Summary: Moved all named days to the same rule, moved all named months to the same rule. Kept same regexes, just consolidated them.
Reviewed By: patapizza
Differential Revision: D5637061
fbshipit-source-id: e08ecf9
Summary: Changed ruleIntervalMonthDDDD to use the ordinal predicate instead of ugly regex
Reviewed By: patapizza
Differential Revision: D5628188
fbshipit-source-id: 1dbe195
Summary: Added EN rule "ruleIntervalFromDDDDMonth" to support "from 10 to 16 August". Used "isDOMValue" helper rather than regex.
Reviewed By: patapizza
Differential Revision: D5610623
fbshipit-source-id: 00a5208
Summary: 'nie' means 'no' in Polish, and isn't a common abbreviation for 'niedziela' (Sunday).
Reviewed By: blandinw
Differential Revision: D5587036
fbshipit-source-id: bfda7fc
Summary:
Fixes#65.
* fixes US holidays
* Black Friday is actually the first day after Thanksgiving day (not necessary the fourth Friday of November)
Reviewed By: JonCoens
Differential Revision: D5533906
fbshipit-source-id: 1824cba
Summary:
In French, the form "at hh" is not valid (it requires an hour indicator).
This fixes false positives such as in "John a un rendez-vous."
Fixes https://github.com/wit-ai/wit/issues/666.
Reviewed By: JonCoens
Differential Revision: D5530713
fbshipit-source-id: ecee1e5
Summary:
* `Duration` before/after `Time` now resolves with the lowest grain
* "now" has an undefined grain `NoGrain`, as depending on the context it might mean different things, as opposed to "right now"
Before:
`day after tomorrow` -> `day` grain
`1 day after tomorrow` -> `hour` grain
Given that the reference date/time is `2013-02-12T04:30:00`.
`one year from now` -> `2014-02-01T00:00:00` with `month` grain.
`one year from today` -> `2014-02-01T00:00:00` with `month` grain.
After:
`day after tomorrow` -> `day` grain
`1 day after tomorrow` -> `day` grain
`one year from now` -> `2014-02-12T04:30:00` with `month` grain (remains the same).
`one year from today` -> `2014-02-12T00:00:00` with `day` grain.
For other `Time` entities involving `Duration`, such as "in + `Duration`", the behavior remains the same: shift to the lower grain (the intent is not precise).
Reviewed By: l5t, blandinw
Differential Revision: D5467164
fbshipit-source-id: b63b6a4
Summary:
MM/YY is a common format for dates in India,UK and other parts of the world.Have added testcases in `Time/EN/corpus.hs` ,however it conflicts with one of the original(2/15 is output now as Feb. 2015 and not the 15th of February).
Closes https://github.com/facebookincubator/duckling/pull/59
Reviewed By: niteria
Differential Revision: D5455881
Pulled By: patapizza
fbshipit-source-id: 23b73a5
Summary:
Today things like `at single`, `at a few`, `at a couple of` would return a `Time`.
Discussed with blandinw to do this very explicit hack right now until other use cases show up.
Reviewed By: niteria
Differential Revision: D5325369
fbshipit-source-id: aec0402
Summary:
This PR contains various smaller but - at least on my data - important performance improvements for matching of German time and time range expressions.
I evaluated this on approx 11.000 time and time range expressions taken from emails (rather formal business travel requests) that have been manually annotated with the "true" time. Comparing this branch to the current master (`d6f8dd`) I get e.g. approx. 80% of the duckling results within +/- 1h of the true value (hours are the smallest grain in my data), vs. only 70% in the master. Other indicators I checked (time/range confusion, other thresholds, failures to find anything in the first place, etc.) were all improved as well.
**Changes**:
* [significant performance plus] added a rule `ruleDateDateInterval` that handles variations of "13.-15.10." correctly. Here the common case is that "13." refers to "13.10." and not "13.CURRENTMONTH". I didn't see an obvious way to fix that in the `<datetime> - <datetime>` rule.
* [significant performance plus] In `ruleMmdd` (which matches expressions like "13.03." in German), I made the last dot optional. At least in less formal text this is quite common to be forgotten. Also here and in `ruleDateDateInterval` I changed the order of the terms in the regular expression matching the month to prefer matching e.g. "10" over matching "1"+"no dot".
* [minor] treat "14/15Uhr" the same as "14-15Uhr"
* [minor] Extended "bis" to also match "bis zum" and "auf den" (e.g. in "von Montag bis zum Freitag" or "von Dienstag auf Mittwoch")
* [minor] Changed `hh:mm` matching to also get the rather esoteric expression "17h00" - should do no harm.
Closes https://github.com/facebookincubator/duckling/pull/54
Reviewed By: blandinw
Differential Revision: D5301815
Pulled By: patapizza
fbshipit-source-id: 8766caf
Summary:
e.g. "New York from 10-6 to 10-22" currently extracts: HH-MM. Instead, it should extract mm-dd i.e. October 10th to October 22nd.
Closes https://github.com/facebookincubator/duckling/pull/48
Reviewed By: niteria
Differential Revision: D5292473
Pulled By: patapizza
fbshipit-source-id: 04f1a4b
Summary:
'one' is a latent time of day.
Restricting a couple of rules to accept non-latent time tokens.
Reviewed By: blandinw
Differential Revision: D5293972
fbshipit-source-id: 07cdb9b
Summary:
The existing "mm/dd" rules only accepts format like "05/27"; However, in practice there might be extra spaces like "05 / 27", "05/ 27". The pull requests tweaks the regex to accept extra space.
Closes https://github.com/facebookincubator/duckling/pull/31
Reviewed By: niteria
Differential Revision: D5147118
Pulled By: patapizza
fbshipit-source-id: f6a5069
Summary:
Add the [Cantonese](https://en.wikipedia.org/wiki/Cantonese) (the official spoken language used in Hong Kong) support to date time
- updated Duration ZH corpus
- updated Time ZH rules and corpus
- updated TimeGrain ZH rules
Closes https://github.com/facebookincubator/duckling/pull/24
Reviewed By: patapizza
Differential Revision: D5143947
Pulled By: niteria
fbshipit-source-id: 9107d05
Summary:
* In DE `frühestens` and `spätestens` act implicitly as `nach` and `vor` (after and before) on times and may also appear after the time
* The rule `ruleTimeofdayTimeofdayInterval` does match `9Uhr-10` but not the
way more common expression `9-10Uhr`; added the same rule with the
second time as non-latent; actually I am not sure whether the original
rule makes sense at all
* Simple extension of `intersect by ,` to THE formal way in DE to express
a date (i.e. `Freitag, der 13.03.2013`)
General remark: I used UTF-8 characters albeit I saw that the other rules and examples use escaped hex encoding for e.g. German umlaute. If there is any reason to do that (it is not very readable), I will of course change that.
Closes https://github.com/facebookincubator/duckling/pull/19
Reviewed By: niteria
Differential Revision: D5070052
Pulled By: patapizza
fbshipit-source-id: 990ad08
Summary:
This is analogous to
[Duckling] Don't produce trivially empty Tokens
but that change did that for intersect, this one
deals with interval.
Reviewed By: patapizza
Differential Revision: D5039215
fbshipit-source-id: 95bd821
Summary:
We already disallowed shallowly-nested intervals.
Interval of an intersection of an interval also seems
unlikely to produce anything useful.
For an input like:
"2016-Jul-29 07:00 - 2016-Jul-29 09:00 UTC"
it goes from:
```
(1.77 secs, 1,095,200,736 bytes)
```
to:
```
(1.33 secs, 857,167,480 bytes)
```
That's -25% time and -22% allocations.
Reviewed By: patapizza
Differential Revision: D5037492
fbshipit-source-id: 481dcdd
Summary:
I've found it useful for debugging to find out where
the EmptyPredicates are coming from.
This allows to change the code to track origin with minimal
amount of change.
One interesting subtlety is that we don't want to
create a new EmptyPredicate if one already exists
Reviewed By: patapizza
Differential Revision: D5037299
fbshipit-source-id: e2a2f5d
Summary:
I find it useful for debugging and adding it in
every debugging session is getting old.
Reviewed By: JonCoens
Differential Revision: D5037181
fbshipit-source-id: 7804cc5
Summary:
Time dimension for Hebrew.
Commented out the failing tests that actually also fail in Clojure.
Reviewed By: JonCoens
Differential Revision: D4970308
fbshipit-source-id: b455142
Summary:
This diff was generated by running `hsclimps`
PLEASE TAKE ONE OF THE FOLLOWING ACTIONS AS SOON AS POSSIBLE:
1) Select Accept and Ship to land this change
2) If you have issues with this diff, request changes
3) If you are no longer the owner, add reviewers and update the `.context` file with the appropriate owner
NOTE: If the diff is unable to land because of a merge conflict I will automatically update it for you.
#accept2ship
Reviewed By: niteria
Differential Revision: D4937839
fbshipit-source-id: bb3d330
Summary:
`intersectMB` was a name used for the purpose of migrating.
This is the last part of the migration.
Reviewed By: patapizza
Differential Revision: D4906098
fbshipit-source-id: a70af78
Summary:
This continues the work from:
"[Duckling] Don't produce trivially empty Tokens"
All the Rules should use intervalMB from now on.
Reviewed By: patapizza
Differential Revision: D4906072
fbshipit-source-id: 277b961
Summary:
We can detect certain kinds of contradictions sooner,
producing a token with an unresolvable Predicate is wasteful.
For a text like:
```
"Demain apres midi 14h 15 h 16h vendredi 14 a 15h"
```
it could produce 7000 tokens with empty predicates.
After this change it produces none and we get a 4x improvement in
time and 6x improvement in allocations.
Note I only covered `ruleIntersect*` here. I need to do this for
other instances as well.
Reviewed By: JonCoens
Differential Revision: D4871078
fbshipit-source-id: 9f0e7ad
Summary:
So far contradictions from intersection only
propagated through intersection. This change
makes it so that it also propagates through intervals
and lets intervals also generate contradictions.
Reviewed By: patapizza
Differential Revision: D4864160
fbshipit-source-id: 8348267
Summary:
This is a very common pattern (>1k occurrences).
Replacing it with something shorter makes the rules a bit less
boilerplate-y.
Feel free to bikeshed the name, I can easily redo the codemod.
Reviewed By: patapizza
Differential Revision: D4848864
fbshipit-source-id: 7baeee3
Summary:
This makes the code easier to read.
I'm not attached to naming, but this is
standard terminology from topology.
Reviewed By: JonCoens, patapizza
Differential Revision: D4848740
fbshipit-source-id: 79c2c20
Summary:
This is the easiest way to fix it, but talking offline
with Julien, we may need to revisit.
It basically gets rid of time series where we were
producing intervals that are not a multiply of the grain.
Reviewed By: patapizza
Differential Revision: D4841759
fbshipit-source-id: 1c4742a
Summary:
This works around https://github.com/haskell/cabal/issues/4350
If we don't do this files get compiled multiple times
and cabal is unhappy.
Reviewed By: patapizza
Differential Revision: D4782749
fbshipit-source-id: 5bbe425
Summary:
* 'no dia 20' (on the 20)
* Unifying two rules into one, with a day grain
See https://github.com/wit-ai/wit/issues/388
Reviewed By: blandinw
Differential Revision: D4715780
fbshipit-source-id: e990954
Summary:
No need to reinvent the wheel when `dependent-sum` has what we need. I re-export `Some(..)` from `Duckling.Dimensions.Types` to cut down on import bloat.
Instead of a `Read` instance I created a `fromName` function.
Reviewed By: zilberstein
Differential Revision: D4710014
fbshipit-source-id: 1d4e86d
Summary: `DNumber` is a terrible name and was only there because legacy. `Numeral` makes more sense for this dimension, so let's use that instead.
Reviewed By: patapizza
Differential Revision: D4707167
fbshipit-source-id: cd78aa3
Summary:
It is no longer necessary after D4676812 and D4698788.
`"I have 9 am 12 pm 1 pm 2pm 4 pm 3 pm on Saturday"` now works in
less than a second, it used to be 10s.
The test suite also got 3s faster.
Reviewed By: patapizza
Differential Revision: D4701890
fbshipit-source-id: 107a55f
Summary:
This is the next step for:
https://fb.facebook.com/groups/527352907463243/permalink/600056483526218/
This:
* changes the time language to be able to track contradictions (`EmptyPredicate`)
* changes the time language to be able to collect non-contradicting pieces, like month and hour and unify them
* provides an efficient way to convert those pieces into (past,future) time series
* adds AMPM predicate runner - there's a bit of overlap with is12H, but it basically works
* changes a test case that was wrong before
* regenerates classifiers, I'm not sure why they changed exactly
Before:
```
res <- H.io $ let sentence = "10am thurs 4.30 thurs 12pm sat" in (debugTokens sentence $ analyze sentence (testContext {lang = EN}) HashSet.empty)
(15.50 secs, 6,171,188,928 bytes)
res <- H.io $ let sentence = "I have 9 am 12 pm 1 pm 2pm 4 pm 3 pm on Saturday" in (debugTokens sentence $ analyze sentence (testContext {lang = EN}) HashSet.empty)
(110.82 secs, 44,031,569,512 bytes)
```
After:
```
res <- H.io $ let sentence = "10am thurs 4.30 thurs 12pm sat" in (debugTokens sentence $ analyze sentence (testContext {lang = EN}) HashSet.empty)
(1.24 secs, 703,020,912 bytes)
res <- H.io $ let sentence = "I have 9 am 12 pm 1 pm 2pm 4 pm 3 pm on Saturday" in (debugTokens sentence $ analyze sentence (testContext {lang = EN}) HashSet.empty)
(9.51 secs, 5,891,109,592 bytes)
```
Reviewed By: JonCoens
Differential Revision: D4676812
fbshipit-source-id: 9810203
Summary:
* we weren't checking the right reference time in `takeNth` and `takeN`
* fixing resulting failing tests for `IT`
* `analyzedNTest` to check that input results in `n` parsed tokens
Reviewed By: niteria
Differential Revision: D4698788
fbshipit-source-id: 2cd4762
