Summary:
Some changes were originally suggested by me during the review of
https://github.com/facebook/duckling/pull/474.
Others are new.
1. "Day after tomorrow/before yesterday"
2. Ordinals in the form of number+suffix like "8th of March"
3. Tuesdays require a special preposition.
4. Support "Yo" (U+0451) in "fourth" and "during daylight".
5. Support special perposition for "next week".
6. Support "one before last" adjective for time grains.
7. Proper suffixes for "quarter" grain.
8. Support "at midnight".
9. Support alternative flag for "afternoon".
Changes in Ordinal and TimeGrain are all driven by the new examples in the
corpus for Time.
There are also a couple of bugfixes:
1. A hidden latin "e" was present in an otherwise Cyrillic regex.
2. Wrong order of options in a regex separated with "|" prevented some matches.
Reviewed By: haoxuany
Differential Revision: D32311714
fbshipit-source-id: 084f6c3893eb5bfd767c267f558b910c6854eb59
Summary:
The Dutch numeral and ordinal rules contained a spelling mistake: [‘achttien’](https://en.wiktionary.org/wiki/achttien) (18) was spelt as ‘achtien’ (note the single *t*).
This PR fixes this spelling mistake everywhere in the code base.
Pull Request resolved: https://github.com/facebook/duckling/pull/602
Reviewed By: patapizza
Differential Revision: D27859001
Pulled By: chessai
fbshipit-source-id: 8da0d8b099d49cf6207d4066cee1fc7da68a418e
Summary:
1. Fix a bug for "fünfter"
3. support for "am ersten Dezember" to " am einunddreißigsten Dezember"
3. duckling recognizes "der fünfte Dezember"
- [x] ``README.md`` is up to date with docker
- [x] ``docker-compose.yml`` is included
- [x] all ``apt-get`` commands are in a single RUN command to avoid multiple stages
- [x] dockerfile now use the files from the directory
- [ ] more tests should be include
Pull Request resolved: https://github.com/facebook/duckling/pull/362
Reviewed By: patapizza
Differential Revision: D15100843
Pulled By: haoxuany
fbshipit-source-id: 407c8bb2a05a1d0cee81c407e81d882fca63a2be
Summary:
- Improved understanding of past periods and dates which include days, months and years.
- Added support for quarters.
- Fixed unnecessary two or more results on some queries.
Pull Request resolved: https://github.com/facebook/duckling/pull/299
Differential Revision: D13878236
Pulled By: patapizza
fbshipit-source-id: 5ae4ac96dd5d75f54ed1567017a81e5a9c8bcc83
Summary:
Add support for ordinals from 13..99
Add support for quarter expressions like:
- próximo trimestre
- quarto trimestre de 2018
- último trimestre de 2015
- de primeiro trimestre de 2017 até terceiro trimestre de 2017
- entre terceiro mês de 2017 e nono mês de 2017
Add new vocabulary for last expressions like:
- últimos 2 meses
- 2 meses anteriores
Pull Request resolved: https://github.com/facebook/duckling/pull/291
Reviewed By: chinmay87
Differential Revision: D13553884
Pulled By: patapizza
fbshipit-source-id: 330af62bf457ffaab65636b29fb7bb51ba8a3ae5
Summary:
Hello.
I added 3 new dimensions: Ordinal, Quantity, & Temperature. And I also added more data to Numeral dimension.
Thank you!
Pull Request resolved: https://github.com/facebook/duckling/pull/251
Reviewed By: beauby
Differential Revision: D9656646
Pulled By: patapizza
fbshipit-source-id: f17797be811d58b0b5bcd02b83c0a699650c9f6e
Summary:
Support custom dimensions
Had to move the definition of `Dimension` from `Duckling.Dimensions.Types` to `Duckling.Types` to avoid cyclic imports between these two modules.
A sample custom dimension is in `exe/CustomDimensionExample.hs`.
Limitations of custom dimensions:
- All rules for a custom dimension must be in the same module with the definition of the custom dimension. Otherwise there will be cyclic imports, because the definition of the dimension and the rules refer to each other.
- The custom dimension must be specified when using `parse`, since there's no way to get all the existing custom dimensions.
Reviewed By: patapizza
Differential Revision: D7630360
fbshipit-source-id: 30e12dcb33611f5692c4f5949de377bf61b75e1e
Summary: Use `HashMap.lookup` to replace the guard match for performance optimization for ordinal rules in Russian.
Reviewed By: chinmay87
Differential Revision: D7372645
fbshipit-source-id: 20555c352a3f65387e0630f5df65c0826af389f1
Summary: Add an option to return latent time entities. This can be used when one is pretty certain that the input contains a datetime.
Reviewed By: patapizza
Differential Revision: D7254245
fbshipit-source-id: e9e0503cace2691804056fcebdc18fd9090fb181
Summary: Added ordinal support for Hindi
Reviewed By: patapizza
Differential Revision: D6944156
fbshipit-source-id: eb5da698e5cccde9a1cc31adf7bc433b89e07454
Summary:
Adding more support for "natural" language compound ordinals:
"twenty nine", "twentynine", and "twenty–nine".
These tend to come up in text that has been transcribed from audio, informal user input, and more formal printed material respectively.
Closes https://github.com/facebook/duckling/pull/152
Reviewed By: panagosg7
Differential Revision: D6850514
Pulled By: patapizza
fbshipit-source-id: 1d3491655cfc1fb3faab0c3e196de1b2361ae95e
Summary:
Fixed 3rd ordinal which didn't work for feminine and neuter genders in nominative.
Fixed when ordinals from 1 to 19 not recognized when used with capital letters.
Added support for ordinals in genitive for all genders.
Closes https://github.com/facebook/duckling/pull/141
Reviewed By: patapizza
Differential Revision: D6740190
Pulled By: panagosg7
fbshipit-source-id: a6a676d95890a7e156346f479c84a71c1b2bd17f
Summary:
Hi, in this pr:
* Added time dimension to Arabic language, thanks to Hussein-Dahir & Yazeed-Obaid for writing time corpus.
* Fixed some bugs in numeral & ordinals and added more test cases for them.
Also, I don't really understand why do we use classifiers in time dimension?
Closes https://github.com/facebook/duckling/pull/123
Reviewed By: blandinw
Differential Revision: D6583313
Pulled By: patapizza
fbshipit-source-id: f7acdef0c032d7b7fd7d224832fdaf484d2df825
Summary:
- Add more tests to Ar ordinal corpus
- Add HashMap lookups and add more rules to composite numbers
Closes https://github.com/facebook/duckling/pull/100
Reviewed By: JonCoens
Differential Revision: D6254264
Pulled By: patapizza
fbshipit-source-id: 6d50286f6bd3a60e34b0eb855fb38e044c3c39dc
Summary:
* Locales support for the library, following `<Lang>_<Region>` with ISO 639-1 code for `<Lang>` and ISO 3166-1 alpha-2 code for `<Region>` (#33)
* `Locale` opaque type (composite of `Lang` and `Region`) with `makeLocale` smart constructor to only allow valid `(Lang, Region)` combinations
* API: `Context`'s `lang` parameter has been replaced by `locale`, with optional `Region` and backward compatibility.
* `Rules/<Lang>.hs` exposes
- `langRules`: cross-locale rules for `<Lang>`, from `<Dimension>/<Lang>/Rules.hs`
- `localeRules`: locale-specific rules, from `<Dimension>/<Lang>/<Region>/Rules.hs`
- `defaultRules`: `langRules` + specific rules from select locales to ensure backward-compatibility
* Corpus, tests & classifiers
- 1 classifier per locale, with default classifier (`<Lang>_XX`) when no locale provided (backward-compatible)
- Default classifiers are built on existing corpus
- Locale classifiers are built on
- `<Dimension>/<Lang>/Corpus.hs` exposes a common `corpus` to all locales of `<Lang>`
- `<Dimension>/<Lang>/<Region>/Corpus.hs` exposes `allExamples`: a list of examples specific to the locale (following `<Dimension>/<Lang>/<Region>/Rules.hs`).
- Locale classifiers use the language corpus extended with the locale examples as training set.
- Locale examples need to use the same `Context` (i.e. reference time) as the language corpus.
- For backward compatibility, `<Dimension>/<Lang>/Corpus.hs` can expose also `defaultCorpus`, which is `corpus` augmented with specific examples. This is controlled by `getDefaultCorpusForLang` in `Duckling.Ranking.Generate`.
- Tests run against each classifier to make sure runtime works as expected.
* MM/DD (en_US) vs DD/MM (en_GB) example to illustrate
Reviewed By: JonCoens, blandinw
Differential Revision: D6038096
fbshipit-source-id: f29c28d
Summary:
We noticed that using UTF-8 characters directly in regexes work.
Hence converting back the escaped characters for readability and maintenance.
Reviewed By: blandinw
Differential Revision: D5787146
fbshipit-source-id: e5a4b9a
Summary:
There are problems in the ordinal recognition for Swedish. The most severe one is that all the numbers above 15 are actually Danish, not Swedish. Apart from that digits and digits followed by a dot are considered ordinals.
This pull request fixes this and also adds support for ordinals up to 100. The structure of the code is similar to in the ordinal recognition in English. Tests are also updated, both the ordinal tests and the time tests where incorrect ordinals were used.
Closes https://github.com/facebookincubator/duckling/pull/86
Reviewed By: JonCoens
Differential Revision: D5698145
Pulled By: patapizza
fbshipit-source-id: c31d7bc
Summary:
The numerical ordinal matching rule in DE is too broad. An ordinal like "1." may not be proceeded or followed by numbers.
* Added negative lookbehind - avoids matching the first "1." in "1.1" as an ordinal.
* Added negative lookahead - avoids matching the second "1." in "1.1. as an ordinal
Closes https://github.com/facebookincubator/duckling/pull/18
Reviewed By: patapizza
Differential Revision: D5069200
Pulled By: niteria
fbshipit-source-id: 0583076
Summary: converting large regex lookups to hashmap lookups in Duckling/Numeral/FR/Rules.hs and Duckling/Ordinal/FR/Rules.hs
Reviewed By: patapizza
Differential Revision: D4836336
fbshipit-source-id: 2241a3a
Summary:
This works around https://github.com/haskell/cabal/issues/4350
If we don't do this files get compiled multiple times
and cabal is unhappy.
Reviewed By: patapizza
Differential Revision: D4782749
fbshipit-source-id: 5bbe425
Summary: Use HashMaps to speed up string pattern matching for UK (Ukranian).
Reviewed By: patapizza
Differential Revision: D4747195
fbshipit-source-id: e582dba
Summary:
No need to reinvent the wheel when `dependent-sum` has what we need. I re-export `Some(..)` from `Duckling.Dimensions.Types` to cut down on import bloat.
Instead of a `Read` instance I created a `fromName` function.
Reviewed By: zilberstein
Differential Revision: D4710014
fbshipit-source-id: 1d4e86d
Summary: `DNumber` is a terrible name and was only there because legacy. `Numeral` makes more sense for this dimension, so let's use that instead.
Reviewed By: patapizza
Differential Revision: D4707167
fbshipit-source-id: cd78aa3
