Commit Graph

740 Commits

Author SHA1 Message Date
Steven Troxler
6db071069b Numeral/ES: Style fixes - names, order, etc
Summary:
Whle looking into fixing https://github.com/facebook/duckling/issues/380
I was having a bit of trouble navigating the existing rules and guessing
what is / is not supported.

This diff refactors the Numeral/ES code to be easier to navigate:
- rename all the `ruleNumeral{1,2,3,4,5,6}` rules to be descriptive
- changes the order to be themed from small to large numbers, and
  make sure the order of defines matches the order of rules at the end
  of the module
- use [20 .. 90] instead of manually specifying the same list out-of-order

Reviewed By: chessai

Differential Revision: D27858134

fbshipit-source-id: b13983d75b36bb4e2b387ef06fe61066d81ae19a
2021-04-23 09:48:06 -07:00
Steven Troxler
bf696ba185 Time/EN: Allow dashes in spelled-out times
Summary:
It's common to use dashes when spelling out times longhand,
e.g. "five-thirty am", but Duckling wasn't handling this at all.

This commit adds rules for times spelled out with dashes. The
rules explicitly forbid the second of the two times from including
digits via a negative match. This is because
- it wouldn't be at all idomatic to write five-26 or five-oh-6
- allowing that pattern clashes with time range parsing, e.g.
  "9-10 am" should parse as a time range, not as "9:10 am"

Reviewed By: chessai

Differential Revision: D27848428

fbshipit-source-id: dfe8b98cb38119a16db2a19db47fd3128783e617
2021-04-22 11:47:28 -07:00
Steven Troxler
23ec021b07 Duration/RU: For non-int times, use the coarsest grain
Summary:
This commit fixes #111, which was an open issue that any non-integer
multiple of any unit of time was being converted to seconds.

My solution is to write a recursive function `Duration.Helpers.inCoarsestGrain`
which, given a grain `g` and double value `v` finds the coarses grain `g'` such that
`v * g` - rounded to the nearest seconds - has integral units.

We call this function only in the case of non-integer multiples, and we start our
search from the given grain because nothing coarser would make sense. The code could
actually be slightly more efficient if we started at the next-smallest grain, but
in the interest of clarity I think this is probably better.

Reviewed By: chessai

Differential Revision: D27891439

fbshipit-source-id: b048310963eb71337fd91ab4ef3c840134a76e73
2021-04-21 13:32:38 -07:00
Steven Troxler
f73277c2b0 Duration/EN: fix lint errors
Summary:
Minor changes to silence most linter errors
- remove redundant import
- delete some redundant brackets and tweak bracket locations
- switch . to $ to avoid excessive-point-free-style warnings

Also linted Duration/Types.hs

Reviewed By: chessai

Differential Revision: D27891440

fbshipit-source-id: d84287edafe328aeddd78b781618ec5c41944bd8
2021-04-21 13:32:37 -07:00
Steven Troxler
35532b0b7c Time/EN: Tighten up handling of split times like "five ten"
Summary:
While debugging an attempt to extend our handling of spelled-out
times, I realized that we are being too aggressive in our parsing of
times like "five ten", because we'll parse "five nine" as possibly
meaning "5:09", which isn't something an English speaker would say
(or rather if they did, it's more likely they mean "five (to) six"
or something similar.

Reviewed By: chessai

Differential Revision: D27848429

fbshipit-source-id: 34d783332fd60359ad9b6e7862367453bc93a1d1
2021-04-20 05:31:53 -07:00
Steven Troxler
a250e60cbb Silence many lint errors on Time/{Helpers,Types}.hs
Summary:
This commit gets rid of all the easy-to-fix lint warnings on time helper modules:
- replacing unnecessary `.` with `$`
- Flipping a lambda in a map to an infix operation
- Use `ts` for a list of times, not `series` which produces a pretty confusing naming collision

There are still quite a lot of lint errors related to name masking, which would be challenging to fix without us coming to an agreement about naming conventions.

But at least in my editor, name-masking errors are a lot less visually noisy than other errors (they only highlight the one name) so I don't mind them as much when skimming the code.

Reviewed By: chessai

Differential Revision: D27842198

fbshipit-source-id: 9091e5349657243b61d7ee169d0d06dd2122ac17
2021-04-19 18:16:55 -07:00
Splinter Suidman
933c287854 NL: Fix spelling of ‘achttien’ (18) (#602)
Summary:
The Dutch numeral and ordinal rules contained a spelling mistake: [‘achttien’](https://en.wiktionary.org/wiki/achttien) (18) was spelt as ‘achtien’ (note the single *t*).

This PR fixes this spelling mistake everywhere in the code base.

Pull Request resolved: https://github.com/facebook/duckling/pull/602

Reviewed By: patapizza

Differential Revision: D27859001

Pulled By: chessai

fbshipit-source-id: 8da0d8b099d49cf6207d4066cee1fc7da68a418e
2021-04-19 18:01:33 -07:00
Steven Troxler
586522b649 Numeral: Silence point-free lint errors in Numeral/Helpers.hs
Summary:
While looking at bugfixes for some ES and IT numeral parsing, I was
looking at the Helpers.hs module and figured I'd silence some lint
errors. There are still some name-masking lint issues, but all others
have been fixed.

Reviewed By: chessai

Differential Revision: D27854164

fbshipit-source-id: 5be8d924b033a55608c455074df1c80c8c0019be
2021-04-19 16:47:10 -07:00
jkachmar
e7f320e0c9 Bumps tasty upper bound (< 1.5) (#605)
Summary:
Closes https://github.com/facebook/duckling/issues/600

 ---

I saw the latest 0.2.0.0 release announcement and decided to try building the project locally.

This compiles successfully with `tasty-1.4.1` and `cabal-install` (cf. the  snippet below if you want to see the full set of packages I compiled/tested with).

<details>
  <summary> cabal.project.freeze </summary>

```
active-repositories: hackage.haskell.org:merge
constraints: any.HUnit ==1.6.2.0,
             any.aeson ==1.5.6.0,
             aeson -bytestring-builder -cffi -developer -fast,
             any.ansi-terminal ==0.11,
             ansi-terminal -example,
             any.ansi-wl-pprint ==0.6.9,
             ansi-wl-pprint -example,
             any.array ==0.5.4.0,
             any.assoc ==1.0.2,
             any.attoparsec ==0.13.2.5,
             attoparsec -developer,
             any.base ==4.14.1.0,
             any.base-compat ==0.11.2,
             any.base-compat-batteries ==0.11.2,
             any.base-orphans ==0.8.4,
             any.bifunctors ==5.5.10,
             bifunctors +semigroups +tagged,
             any.binary ==0.8.8.0,
             any.blaze-builder ==0.4.2.1,
             any.bytestring ==0.10.12.0,
             any.bytestring-builder ==0.10.8.2.0,
             bytestring-builder +bytestring_has_builder,
             any.call-stack ==0.3.0,
             any.case-insensitive ==1.2.1.0,
             any.clock ==0.8.2,
             clock -llvm,
             any.colour ==2.3.5,
             any.comonad ==5.0.8,
             comonad +containers +distributive +indexed-traversable,
             any.constraints ==0.12,
             any.constraints-extras ==0.3.1.0,
             constraints-extras +build-readme,
             any.containers ==0.6.2.1,
             any.data-fix ==0.3.1,
             any.deepseq ==1.4.4.0,
             any.dependent-sum ==0.7.1.0,
             any.directory ==1.3.6.0,
             any.distributive ==0.6.2.1,
             distributive +semigroups +tagged,
             any.dlist ==1.0,
             dlist -werror,
             any.extensible-exceptions ==0.1.1.4,
             any.extra ==1.7.9,
             any.filepath ==1.4.2.1,
             any.generic-deriving ==1.14,
             generic-deriving +base-4-9,
             any.ghc-boot-th ==8.10.4,
             any.ghc-prim ==0.6.1,
             any.happy ==1.20.0,
             any.hashable ==1.3.1.0,
             hashable +integer-gmp,
             any.haskell-src-exts ==1.23.1,
             any.hsc2hs ==0.68.7,
             hsc2hs -in-ghc-tree,
             any.indexed-traversable ==0.1.1,
             any.integer-gmp ==1.0.3.0,
             any.integer-logarithms ==1.0.3.1,
             integer-logarithms -check-bounds +integer-gmp,
             any.io-streams ==1.5.2.0,
             io-streams +network -nointeractivetests +zlib,
             any.io-streams-haproxy ==1.0.1.0,
             any.lifted-base ==0.2.3.12,
             any.monad-control ==1.0.2.3,
             any.mtl ==2.2.2,
             any.network ==3.1.2.1,
             network -devel,
             any.network-uri ==2.6.4.1,
             any.old-locale ==1.0.0.7,
             any.optparse-applicative ==0.16.1.0,
             optparse-applicative +process,
             any.parsec ==3.1.14.0,
             any.pretty ==1.1.3.6,
             any.primitive ==0.7.1.0,
             any.process ==1.6.9.0,
             any.random ==1.2.0,
             any.readable ==0.3.1,
             any.regex-base ==0.94.0.1,
             any.regex-pcre ==0.95.0.0,
             any.regex-posix ==0.96.0.0,
             regex-posix -_regex-posix-clib,
             any.rts ==1.0,
             any.scientific ==0.3.6.2,
             scientific -bytestring-builder -integer-simple,
             any.semigroups ==0.19.1,
             semigroups +binary +bytestring -bytestring-builder +containers +deepseq +hashable +tagged +template-haskell +text +transformers +unordered-containers,
             any.snap-core ==1.0.4.2,
             snap-core -debug +network-uri -portable,
             any.snap-server ==1.1.2.0,
             snap-server -build-pong -build-testserver -debug -openssl -portable,
             any.some ==1.0.2,
             some +newtype-unsafe,
             any.splitmix ==0.1.0.3,
             splitmix -optimised-mixer,
             any.stm ==2.5.0.0,
             any.strict ==0.4.0.1,
             strict +assoc,
             any.tagged ==0.8.6.1,
             tagged +deepseq +transformers,
             any.tasty ==1.4.1,
             tasty +clock +unix,
             any.tasty-hunit ==0.10.0.3,
             any.template-haskell ==2.16.0.0,
             any.text ==1.2.4.1,
             any.text-show ==3.9,
             text-show +base-4-9 +new-functor-classes +template-haskell-2-11,
             any.th-abstraction ==0.4.2.0,
             any.th-compat ==0.1.2,
             any.th-lift ==0.8.2,
             any.these ==1.1.1.1,
             these +assoc,
             any.time ==1.9.3,
             any.time-compat ==1.9.5,
             time-compat -old-locale,
             any.timezone-olson ==0.2.0,
             any.timezone-series ==0.1.9,
             any.transformers ==0.5.6.2,
             any.transformers-base ==0.4.5.2,
             transformers-base +orphaninstances,
             any.transformers-compat ==0.6.6,
             transformers-compat -five +five-three -four +generic-deriving +mtl -three -two,
             any.type-equality ==1,
             any.unbounded-delays ==0.1.1.1,
             any.unix ==2.7.2.2,
             any.unix-compat ==0.5.3,
             unix-compat -old-time,
             any.unordered-containers ==0.2.13.0,
             unordered-containers -debug,
             any.uuid-types ==1.0.4,
             any.vector ==0.12.3.0,
             vector +boundschecks -internalchecks -unsafechecks -wall,
             any.wcwidth ==0.0.2,
             wcwidth -cli +split-base,
             any.zlib ==0.6.2.3,
             any.zlib-bindings ==0.1.1.5
index-state: hackage.haskell.org 2021-04-18T19:08:29Z
```

</details>

The test suite fails with the following error:

```
    Numeral Tests
      AF Tests
        Corpus Tests:              FAIL
          tests/Duckling/Testing/Asserts.hs:90:
          empty result on "agt"
```

...but from what I can tell by the last few CI runs some corpus test errors are not unexpected so I figure this _probably_ isn't a regression.

 ---

I avoided messing with the Stackage snapshot since that seemed out-of-scope, but I could rework this PR (or open another) to update it to the latest LTS resolver and/or add a Stackage Nightly resolver as well.

Pull Request resolved: https://github.com/facebook/duckling/pull/605

Reviewed By: girifb

Differential Revision: D27858969

Pulled By: chessai

fbshipit-source-id: 3f60fd28fe8eeb903de40c696c59ae10eedf12e4
2021-04-19 16:02:32 -07:00
chessai
8ce33d81f5 Update CHANGELOG (#610)
Summary:
Recent changes were allowed in without changelog entries (my fault).
This adds entries for those changes.

Pull Request resolved: https://github.com/facebook/duckling/pull/610

Reviewed By: haoxuany

Differential Revision: D27867893

Pulled By: chessai

fbshipit-source-id: 6e3d9dce496f0f9344dcb65c5e2dd3f1f1c380eb
2021-04-19 16:02:32 -07:00
Steven Troxler
9bd4c9b7fb Time/EN: Allow latent match for <part-of-day> <latent-time-of-day>
Summary:
This fixes #592 in a very conservative way: the reason why `ruleIntersect` does
not detect "tonight 815" and "tonight eight fifteen" as it does "tonight 8:15"
is because it explicitly forbids the second part of the intersection from being
latent, unless it is a year.

I don't think it's a good idea to remove the restriction on latent inputs in
`ruleIntersect`, so instead I just made a new rule specifically for the
intersection of `<part-of-day> <time-of-day>`.

It also seems to me that there's a lot of room for this to be too aggressive,
for example if I say "tonight 500 people will laugh" the "tonight" and "500"
aren't really linked. So, I set the rule to be latent; this may be too conservative
to be useful though (do client libraries usually allow latent results?).

Reviewed By: chessai

Differential Revision: D27842596

fbshipit-source-id: 36ac59e31c632d4864241bce291147a46d52f780
2021-04-19 13:05:50 -07:00
evjava
f1cb3bc87c Russian(RU) duration improvements (#375)
Summary:
- dimunitives for minutes and hours
- quarters of an hour
- added 'сутки' (24-hour period)
Pull Request resolved: https://github.com/facebook/duckling/pull/375

Reviewed By: stroxler

Differential Revision: D20332233

Pulled By: chessai

fbshipit-source-id: 479858e6c5de856a6965b6193c481e654a6e04fb
2021-04-16 14:31:41 -07:00
chessai
d712bee78b initialise changelog (#597)
Summary:
resolves https://github.com/facebook/duckling/issues/455

Pull Request resolved: https://github.com/facebook/duckling/pull/597

Reviewed By: girifb

Differential Revision: D27806129

Pulled By: chessai

fbshipit-source-id: 4bca273fef5df0848c2b92d3ac753b66cedfcdcb
2021-04-16 10:02:28 -07:00
chessai
3e4fe974a3 update ci (#594)
Summary:
- Update GHC 8.10.2 to 8.10.3
- Add GHC 9.0.1
- Make stack CI as extensive as cabal CI
- update tested-with
- remove cabal CI, keep stack only

Pull Request resolved: https://github.com/facebook/duckling/pull/594

Reviewed By: girifb

Differential Revision: D27710921

Pulled By: chessai

fbshipit-source-id: 098bcbb33c7dc7dfac4e2e46ca9ea07083b47b59
2021-04-12 18:32:23 -07:00
chessai
e6e000fb62 v0.1.6.1 -> v0.2.0.0 (#593)
Summary:
Necessary major version bump as per PVP: https://pvp.haskell.org/

Pull Request resolved: https://github.com/facebook/duckling/pull/593

Reviewed By: girifb

Differential Revision: D27710929

Pulled By: chessai

fbshipit-source-id: 16fb24e108ab6696d31c94573e79644248bf00e2
2021-04-12 14:17:18 -07:00
leandro.guisandez@pgconocimiento.com
f3e128a07b Add Ordinal dimension to CA (Catalan) locale
Summary: Adds Ordinal rules to Catalan language

Reviewed By: patapizza

Differential Revision: D27681617

Pulled By: chessai

fbshipit-source-id: 145a1117eeff10839484f34a87e9bd685382d42e
2021-04-09 13:05:19 -07:00
leandro.guisandez@pgconocimiento.com
7907812184 Initialise Catalan language with Numeral
Summary: Adds Catalan language and Numeral rules for it

Reviewed By: haoxuany

Differential Revision: D26518604

Pulled By: chessai

fbshipit-source-id: e6b4b0ceb9b7931d086c732dd03fb5cbbe062d5b
2021-04-08 14:47:02 -07:00
chessai
9babaee20a add missing copyright headers (#591)
Summary: Pull Request resolved: https://github.com/facebook/duckling/pull/591

Reviewed By: patapizza

Differential Revision: D27623732

Pulled By: chessai

fbshipit-source-id: e38a64d04d004ea80fabeb5efbc635b00e2802df
2021-04-08 10:46:31 -07:00
Amr Keleg
8a8f557002 Add more variants of egp to EN and AR AmountOfMoney dimension (#590)
Summary:
These are popular variants/abbreviations of Egyptian pounds.
All these forms are documented on wikipedia (https://en.wikipedia.org/wiki/Egyptian_pound)

Pull Request resolved: https://github.com/facebook/duckling/pull/590

Reviewed By: haoxuany

Differential Revision: D27598249

Pulled By: chessai

fbshipit-source-id: 42ae9115b1def48c58e50a6deb624c3407c029f3
2021-04-07 12:31:41 -07:00
Mustafa ALP
3157d2e553 Time Dimension for TR locale (#584)
Summary:
Added time dimension for Turkish language

Pull Request resolved: https://github.com/facebook/duckling/pull/584

Differential Revision: D27235743

Pulled By: chessai

fbshipit-source-id: 7419ff7373d942530f0eb35939acb9970b918672
2021-04-06 10:32:18 -07:00
chessai
4c4d5ca725 bump dependencies (#588)
Summary: Pull Request resolved: https://github.com/facebook/duckling/pull/588

Reviewed By: girifb

Differential Revision: D27435008

Pulled By: chessai

fbshipit-source-id: f0b0a42752cfebcf290cbc6c6d194ba09724670f
2021-03-30 12:47:10 -07:00
Mustafa ALP
5385346e69 Update README.md (#585)
Summary:
Convert `This` to `Seal` in order to make this example working.

Pull Request resolved: https://github.com/facebook/duckling/pull/585

Reviewed By: girifb

Differential Revision: D27235685

Pulled By: chessai

fbshipit-source-id: 71a712a622b5d9d10f7842276a2b8f60f962477e
2021-03-26 14:33:53 -07:00
Steven Troxler
d47cb1b002 Ignore the no-"-1" linter error for duckling
Summary:
By default, Facebook Haskell code has a lint error banning
-1s, because at one point it was used as a default value in some
API handlers and it's better to use Option + Nothing for this use case.

But in Duckling - particularly in the Time module - it's quite
common that we actually want to work with -1 as a meaningful integer
value, involving some kind of offset.

Reviewed By: chessai

Differential Revision: D27118984

fbshipit-source-id: 5fe2200e8005a20855d7fdd3a8eb2ad33291edc8
2021-03-17 10:47:28 -07:00
Steven Troxler
4917d426cf Style tweaks
Summary:
The facebook internal linters prefer us to avoid
excessive point-free style and extra $ where we could
instead move existing brackets.

Making those style tweaks for Time/EN/Rules.hs because
I was looking at the file as part of

Reviewed By: chessai

Differential Revision: D27108042

fbshipit-source-id: 7c8e76578476ea14d655131943e693c5159b12d2
2021-03-17 10:47:28 -07:00
Steven Troxler
78904b6680 Time - #444 Handle 2-digit date in existing d/m/y rule
Summary:
The pattern laied out in the bug report
https://github.com/facebook/duckling/issues/444
is actually already handled by the pattern
`<day-of-month>(ordinal or number)/<named-month>/year`.

The problem is purely that the regular expression doesn't
match 2-digit years, so the pattern is getting skipped rather
than evaluated. This diff fixes the regexp and adds a new example
with a 2-digit pattern.

This fixes the bug report:
```
> debug (makeLocale EN Nothing) "10-Apr-15" [Seal Time]
<day-of-month>(ordinal or number)/<named-month>/year (10-Apr-15)
-- integer (numeric) (10)
-- -- regex (10)
-- regex (-)
-- April (Apr)
-- -- regex (Apr)
-- regex (-)
-- regex (15)
[Entity {dim = "time", body = "10-Apr-15", value = RVal Time (TimeValue (SimpleValue (InstantValue {vValue = 2015-04-10 00:00:00 -0200, vGrain = Day})) [SimpleValue (InstantValue {vValue = 2015-04-10 00:00:00 -0200, vGrain = Day})] Nothing), start = 0, end = 9, latent = False, enode = Node {nodeRange = Range 0 9, token = Token Time TimeData{latent=False, grain=Day, form=Nothing, direction=Nothing, holiday=Nothing, hasTimezone=False}, children = [Node {nodeRange = Range 0 2, token = Token Numeral (NumeralData {value = 10.0, grain = Nothing, multipliable = False, okForAnyTime = True}), children = [Node {nodeRange = Range 0 2, token = Token RegexMatch (GroupMatch ["10"]), children = [], rule = Nothing}], rule = Just "integer (numeric)"},Node {nodeRange = Range 2 3, token = Token RegexMatch (GroupMatch []), children = [], rule = Nothing},Node {nodeRange = Range 3 6, token = Token Time TimeData{latent=False, grain=Month, form=Just (Month {month = 4}), direction=Nothing, holiday=Nothing, hasTimezone=False}, children = [Node {nodeRange = Range 3 6, token = Token RegexMatch (GroupMatch []), children = [], rule = Nothing}], rule = Just "April"},Node {nodeRange = Range 6 7, token = Token RegexMatch (GroupMatch []), children = [], rule = Nothing},Node {nodeRange = Range 7 9, token = Token RegexMatch (GroupMatch ["15"]), children = [], rule = Nothing}], rule = Just "<day-of-month>(ordinal or number)/<named-month>/year"}}]
```

Reviewed By: chessai

Differential Revision: D27106007

fbshipit-source-id: 4751672aef807464febef87f6d22d7270bd335df
2021-03-17 10:35:00 -07:00
mustafaalp43@gmail.com
56fd7b0aaf Feature/Turkish money (#579)
Summary:
Added amount of money dimension for Turkish language
Pull Request resolved: https://github.com/facebook/duckling/pull/579

Test Plan: :test Endpoint.Duckling.Test

Reviewed By: haoxuany, bugra

Differential Revision: D27017300

Pulled By: chessai

fbshipit-source-id: e8cb257a2953675f54269ed358948e8cbe38af7b
2021-03-17 10:35:00 -07:00
chessai
8de488475a Load timezones more leniently. (#582)
Summary:
On some linux systems, such as on NixOS, /usr/share/zoneinfo does not
exist. What does exist in its place is /etc/zoneinfo. So, we should try to load
that if /usr/share/zoneinfo does not exist.

Pull Request resolved: https://github.com/facebook/duckling/pull/582

Reviewed By: girifb

Differential Revision: D27086925

Pulled By: chessai

fbshipit-source-id: f4a38822be9888d57034f67a6f7abd17d56d38b8
2021-03-16 13:04:09 -07:00
Steven Troxler
55168db92f Update classifiers
Summary:
I was testing an unrelated change (which doesn't change
classifier scores) and reran classifiers just to be safe, I noticed
that the scores changed.

This diff updates them.

Reviewed By: chessai

Differential Revision: D26892970

fbshipit-source-id: c7da3e3b7d01955f98b287a3ff4e7c1ff2837c7f
2021-03-08 14:02:45 -08:00
Steven Troxler
fc5278855d Combine duplicated examples
Summary:
I was looking at adding support for "next week" constructions in Spanish to
close https://github.com/facebook/duckling/issues/553 (which it appears has
already been handled), when I noticed that the equivalent logic for English
has been split into two separate examples: "coming week" isn't in the same
example as other equivalent constructs like "upcoming week" and "next week".

This diff combines them, which I think is clearer and fewer lines of code

Reviewed By: chessai

Differential Revision: D26892322

fbshipit-source-id: 68ca4644759198fc79d963ae080495c3f2d4a923
2021-03-08 13:47:06 -08:00
Mustafa ALP
b671d75d02 Typo correction (#574)
Summary:
This commit includes typo correction for `half` and `three` equivelant in Turkish

Pull Request resolved: https://github.com/facebook/duckling/pull/574

Reviewed By: girifb

Differential Revision: D26726718

Pulled By: chessai

fbshipit-source-id: 840c2d8e491057b6ccec81562ff64356789f587d
2021-03-03 12:31:33 -08:00
Aaron Yue
f68ebf808a forward factor parse tree for exploit in T85548324
Summary: due to exploit in T85548324, factoring the input to get a smaller parse tree (the existing one parses tail recursively, whereas this one uses ruleIntersect which is still bad, but slightly better).

Differential Revision: D26657170

fbshipit-source-id: fe3a738073b4d30ae401521bb692f4a4bba48d96
2021-02-24 22:20:20 -08:00
Amit Manchanda
37b671389c add: support for specific times in HI duration (#424)
Summary: Pull Request resolved: https://github.com/facebook/duckling/pull/424

Reviewed By: girifb

Differential Revision: D26411920

Pulled By: chessai

fbshipit-source-id: 3f0063e4786688579f2f53f46b31bda5d222d402
2021-02-17 14:48:07 -08:00
Aleksey Landyrev
590651150b Add Time dimension for RU language
Summary: Used b40e2147a9 as reference

Reviewed By: kappa

Differential Revision: D24773196

Pulled By: chessai

fbshipit-source-id: 7cc008c0ee80f930efd76e39bb16ca91ec94b641
2021-02-12 12:02:44 -08:00
Maurice Döpke
75af12524f adds german time rule for expressions like: Montag in 3 Wochen (#332)
Summary:
closes https://github.com/facebook/duckling/issues/331

Pull Request resolved: https://github.com/facebook/duckling/pull/332

Reviewed By: girifb

Differential Revision: D26283481

Pulled By: chessai

fbshipit-source-id: 054c6467a69896ff3ebbd1f9bc0734aadf1b6dbe
2021-02-09 14:33:37 -08:00
Maurice Döpke
998b13bceb Adds german times rules like "Übernächste Woche" (week after next) (#330)
Summary:
fixes https://github.com/facebook/duckling/issues/329  and allows for recognizing of terms like übernächste woche

Pull Request resolved: https://github.com/facebook/duckling/pull/330

Reviewed By: girifb

Differential Revision: D26284196

Pulled By: chessai

fbshipit-source-id: 160e73668b835c83adb0fd1c396a8a2977e86516
2021-02-09 10:48:32 -08:00
evjava@yandex.ru
ff4a1a5bae Be more permissive with numerals [20, 90]
Summary: There are a handful of more spelling for russian numbers [20, 30 .. 90] that we aren't handling. Additionally, we optimise for recall over precision by allowing some invalid spellings that could be understandable typos.

Reviewed By: patapizza

Differential Revision: D26285711

Pulled By: chessai

fbshipit-source-id: fd8a8f373d228a526e79b22326eff48bb966310d
2021-02-05 15:31:18 -08:00
kcnhk1@gmail.com
3f2f307735 Time - add more common expressions
Summary:
Added:
last <duration>
<time> <day-of-month>

Reviewed By: haoxuany

Differential Revision: D26263977

Pulled By: chessai

fbshipit-source-id: b00ece753593a7fabe45bbaa9e1f013860e38d80
2021-02-04 16:32:11 -08:00
Amit Manchanda
c2b280c9ef add: support for composite duration in hindi (#425)
Summary: Pull Request resolved: https://github.com/facebook/duckling/pull/425

Reviewed By: girifb

Differential Revision: D26263097

Pulled By: chessai

fbshipit-source-id: 29605023746a30dc286ffb246eb30fdc4067cbd8
2021-02-04 16:19:23 -08:00
kcnhk1@gmail.com
9a6aeb9b51 Distance - introduce interval rules
Reviewed By: haoxuany

Differential Revision: D26256269

Pulled By: chessai

fbshipit-source-id: 0c3ca267158fd5189fef5540d5bbb903b0dd00b4
2021-02-04 12:02:32 -08:00
kcnhk1@gmail.com
722cc838ff Volume - extend interval support
Reviewed By: haoxuany

Differential Revision: D26255089

Pulled By: chessai

fbshipit-source-id: e4bdb0aa3c1be55dff0a5577155a3d0469d6762d
2021-02-04 12:02:32 -08:00
kcnhk1@gmail.com
67c1dbe94f AmountOfMoney - extend interval support
Reviewed By: haoxuany

Differential Revision: D26254863

Pulled By: chessai

fbshipit-source-id: dfc06f9831de2d50c11d252429c4fb9b8c1eb13a
2021-02-04 11:19:19 -08:00
kcnhk1@gmail.com
b6da3929ce Extend distance rules
Summary:
Add rules:
- one meter and <dist>
- <dist> meters and <dist>

Reviewed By: girifb

Differential Revision: D26191350

Pulled By: chessai

fbshipit-source-id: 52c85c94647e98fba866c24d3386eea988f7f58c
2021-02-03 15:01:39 -08:00
kcnhk1@gmail.com
776b1ec64d extend AmountOfMoney rules
Summary:
Add rules:
- `hkd` as HKD, and related rules (prefix and suffix)
- dollar and <amount-of-money> rule
- dollar and a half rule
- intersection for <amount-of-money> and `a half`

Changed:
- dime and dollar rules now have improved coverage

Reviewed By: girifb

Differential Revision: D26191724

Pulled By: chessai

fbshipit-source-id: bf63b6eaa751fb96dcf341fa2b66db06a6eeca79
2021-02-03 14:05:30 -08:00
Daniel Cartwright
041a81ad1a Use System.FilePath.Posix
Summary: Results in no change on linux/macos, but this is necessary on windows to prevent paths from being botched

Reviewed By: girifb

Differential Revision: D25893201

fbshipit-source-id: ca79dd8a766aecf27562044865d9bc258a4e8d11
2021-02-03 13:31:34 -08:00
Amr Keleg
e673ba5e84 Quantity/EN: Support k.g k.g. (#570)
Summary:
Adding . in between kilogram units used to be extracted as a Numeral
instead of Quantity.

Pull Request resolved: https://github.com/facebook/duckling/pull/570

Reviewed By: patapizza

Differential Revision: D26199687

Pulled By: chessai

fbshipit-source-id: 65e39f20296946d5762d7180b12878f4e66ea701
2021-02-03 12:46:27 -08:00
kcnhk1@gmail.com
496842d16a Extend numeral rules
Summary:
- Extend fraction rule
- add mixed fraction rules
- add prefix of 10/100/10_000 rules

Reviewed By: girifb

Differential Revision: D26191175

Pulled By: chessai

fbshipit-source-id: c2f6b74602e1b8061e0c556721ad8e36821fdb5c
2021-02-03 11:19:33 -08:00
jfulse
788f63eeac Parse more date formats in Norwegian (#395)
Summary:
In general there are some clashes between time formats `hhmm` and date formats `ddmm`. For example, depending on context, `22.10` can mean clock time ten past ten or the twenty second of october. In general it's correct to interpret this as clock time, as Duckling currently does.

But there are some cases not currently covered by Duckling where we have more unambiguous dates, e.g. `12.03.2018` and `27.11`. These are included here (in addition to midnight `24:00` which was also missing).

#### Changes:

- Bug in `ruleDdmm` regex meant that dates on the format `dd/mm` where `mm > 9` were not parsed
- `ruleYyyymmdd` now also parses dots and forward slashes, i.e. `2012.05.14` and `2012/05/14`
- New rule `rule2400` parses `24:00` and `24.00` (I elected not to include it in `ruleMidnighteodendOfDay` as it has grain minute rather than day)
- New rule `ruleDmm` parses `1/10`, `9.12` etc
- New rule `ruleDDm` parses `10/3`, `11.1` etc
- New rule `ruleDdDotMm` parses `25.02`, `31.10` etc
- `ruleDdmmyyyy` now also parses dots, i.e. `03.10.1983`
- New tests

Pull Request resolved: https://github.com/facebook/duckling/pull/395

Reviewed By: patapizza

Differential Revision: D26193069

Pulled By: chessai

fbshipit-source-id: cf711807fa1d40be2303f2426d74ded40c2e23b3
2021-02-02 23:18:48 -08:00
Maxime Biais
16708d9572 Minor Volume.FR improvement: add "Centilitre" type (#354)
Summary:
Minor Volume.FR improvement: add "Centilitre" type. This is useful for recipe parsing.

Pull Request resolved: https://github.com/facebook/duckling/pull/354

Reviewed By: patapizza

Differential Revision: D26193246

Pulled By: chessai

fbshipit-source-id: ddd551e062b8efeff1e786e30e35815c0c29a34c
2021-02-01 22:48:34 -08:00
kcnhk1@gmail.com
61e06c3aa6 Add initial support for volumes in Chinese
Reviewed By: girifb

Differential Revision: D26183123

Pulled By: chessai

fbshipit-source-id: 1acd27d5172cfb5bccbeb1576700e2c60a8e3907
2021-02-01 16:05:42 -08:00
Igor Kuzmenko
9993911e3b Adds UAH currency Type and examples to EN and RU Corpus (#433)
Summary:
This PR adds UAH currency Type and examples to EN and RU Corpus

Pull Request resolved: https://github.com/facebook/duckling/pull/433

Reviewed By: girifb

Differential Revision: D25102990

Pulled By: chessai

fbshipit-source-id: ed40e8dfcf145a65c7e6d87158da0efacb32e256
2021-02-01 14:32:24 -08:00