Commit Graph

66 Commits

Author SHA1 Message Date
Lautaro Emanuel
e1641aeba4 Added missing 'TR' time test cases (#661)
Summary:
Fixes https://github.com/facebook/duckling/issues/660.

Pull Request resolved: https://github.com/facebook/duckling/pull/661

Test Plan: :test Duckling.Time.TR.Tests

Reviewed By: stroxler

Differential Revision: D32145794

Pulled By: chessai

fbshipit-source-id: 4d55043f133b8238e9e8360264a3fbea6af2d022
2021-11-10 10:34:28 -08:00
Ovidiu Nistor
dd70d80dc1 Add Japanese time dimension (#646)
Summary:
Add the most common rules for Japanese time dimension.

Pull Request resolved: https://github.com/facebook/duckling/pull/646

Reviewed By: stroxler

Differential Revision: D30675005

Pulled By: chessai

fbshipit-source-id: 917aa98b5cfe0c73d207b1f51b80d8e17a1c7e6a
2021-09-21 12:28:06 -07:00
chessai
99e1dce9c4 restrict dimensions to only those specified (#625)
Summary:
Resolves https://github.com/facebook/duckling/issues/624

Before patch (specifying quantity and numeral, but time still shows up):
```
❯ curl -XPOST http://0.0.0.0:8000/parse --data 'locale=en_US&text="June 21 and 3 cups of sugar"&dims="[\"quantity\",\"numeral\"]"' | jq
[
  {
    "body": "June 21",
    "start": 1,
    "value": {
      "values": [
        {
          "value": "2021-06-21T00:00:00.000-07:00",
          "grain": "day",
          "type": "value"
        },
        {
          "value": "2022-06-21T00:00:00.000-07:00",
          "grain": "day",
          "type": "value"
        },
        {
          "value": "2023-06-21T00:00:00.000-07:00",
          "grain": "day",
          "type": "value"
        }
      ],
      "value": "2021-06-21T00:00:00.000-07:00",
      "grain": "day",
      "type": "value"
    },
    "end": 8,
    "dim": "time",
    "latent": false
  },
  {
    "body": "3 cups of sugar",
    "start": 13,
    "value": {
      "value": 3,
      "type": "value",
      "product": "sugar",
      "unit": "cup"
    },
    "end": 28,
    "dim": "quantity",
    "latent": false
  }
]
```

After patch (time no longer shows up):
```
❯ curl -XPOST http://0.0.0.0:8000/parse --data 'locale=en_US&text="June 21 and 3 cups of sugar"&dims="[\"quantity\",\"numeral\"]"' | jq
[
  {
    "body": "3 cups of sugar",
    "start": 13,
    "value": {
      "value": 3,
      "type": "value",
      "product": "sugar",
      "unit": "cup"
    },
    "end": 28,
    "dim": "quantity",
    "latent": false
  }
]
```

Pull Request resolved: https://github.com/facebook/duckling/pull/625

Reviewed By: stroxler

Differential Revision: D28851759

Pulled By: chessai

fbshipit-source-id: d3b3f33092c7e60bf29886939488ed562a213c35
2021-06-03 10:33:42 -07:00
leandro.guisandez@pgconocimiento.com
5d8d99bbf4 Init
Summary: Initialise Time for CA (Catalan) language

Reviewed By: stroxler

Differential Revision: D28455273

Pulled By: chessai

fbshipit-source-id: be9a4d61692ba4fb32986e161e9fdd6d25a357dc
2021-05-18 13:50:19 -07:00
Steven Troxler
0efbfe5988 Give a usable error if we fail to parse reftime parameter
Summary:
My attention was brought to this issue by the linter complaining about `read`.

The linter is just as happy with `error`, but I do think it's better to fail
with a usable error message here than with whatever error `read` gives us.

Reviewed By: chessai

Differential Revision: D28213406

fbshipit-source-id: f101d0515ee64978480bdbb873ff72d80d124969
2021-05-07 06:20:03 -07:00
Steven Troxler
eba5d0a825 Simple style fixes for outer layers around Engine.hs
Summary:
Easy style fixes for ExampleMain.hs, Debug.hs, Api.hs, Core.hs

Most of these are just lint fixes, but I also made a few not-just-lint changes
to conform to some elements of our style guide that I agree with:
- if the type signature doesn't fit on one line, then put one type per line
  with nothing on the first line, so that all types are vertically aligned - makes
  for a quick skim
- try to avoid mixing same-line function args with hanging function args: hang
  all arguments or none at all to get a more outline-like feel, again better for
  skimming

I was actually able to eliminate all errors for most of these modules - the name
collisions I usually give up on were manageable by hiding + easy variable renames

Reviewed By: chessai

Differential Revision: D28213246

fbshipit-source-id: 1f77d56f2ff8dccfd5f3b534f087c07047b92885
2021-05-06 08:54:56 -07:00
leandro.guisandez@pgconocimiento.com
7907812184 Initialise Catalan language with Numeral
Summary: Adds Catalan language and Numeral rules for it

Reviewed By: haoxuany

Differential Revision: D26518604

Pulled By: chessai

fbshipit-source-id: e6b4b0ceb9b7931d086c732dd03fb5cbbe062d5b
2021-04-08 14:47:02 -07:00
Mustafa ALP
3157d2e553 Time Dimension for TR locale (#584)
Summary:
Added time dimension for Turkish language

Pull Request resolved: https://github.com/facebook/duckling/pull/584

Differential Revision: D27235743

Pulled By: chessai

fbshipit-source-id: 7419ff7373d942530f0eb35939acb9970b918672
2021-04-06 10:32:18 -07:00
chessai
8de488475a Load timezones more leniently. (#582)
Summary:
On some linux systems, such as on NixOS, /usr/share/zoneinfo does not
exist. What does exist in its place is /etc/zoneinfo. So, we should try to load
that if /usr/share/zoneinfo does not exist.

Pull Request resolved: https://github.com/facebook/duckling/pull/582

Reviewed By: girifb

Differential Revision: D27086925

Pulled By: chessai

fbshipit-source-id: f4a38822be9888d57034f67a6f7abd17d56d38b8
2021-03-16 13:04:09 -07:00
Aleksey Landyrev
590651150b Add Time dimension for RU language
Summary: Used b40e2147a9 as reference

Reviewed By: kappa

Differential Revision: D24773196

Pulled By: chessai

fbshipit-source-id: 7cc008c0ee80f930efd76e39bb16ca91ec94b641
2021-02-12 12:02:44 -08:00
Daniel Cartwright
041a81ad1a Use System.FilePath.Posix
Summary: Results in no change on linux/macos, but this is necessary on windows to prevent paths from being botched

Reviewed By: girifb

Differential Revision: D25893201

fbshipit-source-id: ca79dd8a766aecf27562044865d9bc258a4e8d11
2021-02-03 13:31:34 -08:00
Tobias Wochinger
97636f525e skip logfile creation if no logging (#377)
Summary:
**Motivation**
Currently the log files and the log directory for the server are always created, even if the logging is disabled. If duckling is used on OpenShift the file creation leads to errors if no volume mount is defined.

**Proposed Change**:
Only create log files / log directory if the logging is enabled.

Pull Request resolved: https://github.com/facebook/duckling/pull/377

Reviewed By: patapizza

Differential Revision: D26148878

Pulled By: chessai

fbshipit-source-id: f8e2b1a38586121d854a4826c322b4b859cc9c6b
2021-01-29 11:32:32 -08:00
chaitu9701
28cb5ebd2a Adding Numerical Dimention support for Telugu language (#470)
Summary:
This pull request is to add support for Telugu language (Numerical Dimension) to Duckling

Pull Request resolved: https://github.com/facebook/duckling/pull/470

Differential Revision: D25546700

Pulled By: chessai

fbshipit-source-id: 1d88ee27da8a577a4a79ff31be8cb55ed6444c4e
2020-12-15 17:48:03 -08:00
chessai
a319da07b2 ExampleMain: fix build failure (#560)
Summary: Pull Request resolved: https://github.com/facebook/duckling/pull/560

Reviewed By: patapizza

Differential Revision: D25564850

Pulled By: chessai

fbshipit-source-id: 631f96a3ed71b9d7707560ff6bfe7596feee2305
2020-12-15 11:48:04 -08:00
Daniel Cartwright
17f11135f2 Document how to pass dimensions to the example application
Summary: External users are repeatedly confused by lack of results from the duckling example executable. We should just go through all dimensions for the duckling call in the example app.

Reviewed By: patapizza

Differential Revision: D25468199

fbshipit-source-id: 6cf56b130d4d0aa3181f098d6a7c9a133bfa85ff
2020-12-14 15:02:37 -08:00
Sajjad Heydari
700118644c FA Setup (#520)
Summary: Pull Request resolved: https://github.com/facebook/duckling/pull/520

Reviewed By: patapizza

Differential Revision: D25072459

Pulled By: chessai

fbshipit-source-id: 5db72eda36fe166a452b2345cab75fb1508b192b
2020-11-19 12:20:00 -08:00
Josef Svenningsson
7889f396f3 Remove dependency on Data.Some (#533)
Summary:
Pull Request resolved: https://github.com/facebook/duckling/pull/533

In recent versions of Data.Some the name of the constructor, `This` has changed name to `Some`. This has become rather problematic for us to migrate so we're just going to remove the dependency. The meat of this diff is adding the type `Seal` to `Duckling.Types`. That type replaces `Some`.

Reviewed By: pepeiborra

Differential Revision: D23929459

fbshipit-source-id: 8ff4146ecba4f1119a17899961b2d877547f6e4f
2020-09-28 01:33:01 -07:00
Jesse Vermeulen
922d06b6e4 AF Setup + Numeral (#422)
Summary:
- Setup Afrikaans (AF) language
- Added Numeral Dimension

Some of the paths have changed, and some extra files were necessary, after
basing initial work off 24d3f19976

I followed some of the Numeral examples from Dutch as well as Hungarian,
since Afrikaans and Dutch have some similarities.

One thing was examples for numbers having the number as an example, which I
didn't do here, because I'm not sure it's necessary.
Pull Request resolved: https://github.com/facebook/duckling/pull/422

Reviewed By: awalterschulze

Differential Revision: D18348617

Pulled By: patapizza

fbshipit-source-id: b8c4218629c264b48d6f2cecc4c23e2e281a64da
2020-01-10 15:02:50 -08:00
Ondrej Kováč
294771593d Added Slovak (sk) language with numeral dimension and tests.
Summary: Pull Request resolved: https://github.com/facebook/duckling/pull/428

Reviewed By: haoxuany

Differential Revision: D18348514

Pulled By: patapizza

fbshipit-source-id: 9b0b9c2caa9fec8330746059eefa6185a8f3e072
2020-01-10 14:49:08 -08:00
Aleksandar Hitrov
cff1ca0080 Add Time dimension for language BG
Summary: Pull Request resolved: https://github.com/facebook/duckling/pull/403

Reviewed By: haoxuany

Differential Revision: D18348752

Pulled By: patapizza

fbshipit-source-id: ce3b5c76cb2cf39114216842529d4eaa8df5b93f
2020-01-10 14:34:22 -08:00
Kamolsit Mongkolsrisawat
7f2450e067 Add Numeral dimension for new language TH (#399)
Summary:
Hello,
I am new to Haskell, but I would like to add Thai language (TH) to Duckling.
I have tried to extended Duckling by adding Numeral dimension for new language TH.
Please have a look at it and see what we can improve.

Thanks!
Pull Request resolved: https://github.com/facebook/duckling/pull/399

Reviewed By: patapizza

Differential Revision: D17651508

Pulled By: haoxuany

fbshipit-source-id: 4b3ee1352f239eee637958f5e9dce68430352a0a
2019-11-27 15:48:38 -08:00
Julien Odent
bf89e34365 Relicense to BSD3
Reviewed By: JoelMarcey

Differential Revision: D15439223

fbshipit-source-id: c5af3cb06318748142fe503945b38beffadfc28a
2019-05-22 10:46:39 -07:00
akki2825
a2494caf61 add Kannada support (for Numeral)
Summary: Pull Request resolved: https://github.com/facebook/duckling/pull/322

Reviewed By: patapizza

Differential Revision: D13888983

Pulled By: haoxuany

fbshipit-source-id: b7a3b5c72dae0ccc637cf6faa14670ebf868bb50
2019-02-08 11:31:39 -08:00
Hamza Kwisaba
392e584d1b swahili numerical support
Summary: Adding support for swahili numerals

Reviewed By: patapizza

Differential Revision: D13637092

fbshipit-source-id: 6fe0facfa74caae6fd00e0e84b09571aca616f21
2019-01-18 01:16:13 -08:00
Julien Odent
49e0ffef04 Time/UK: Remove redundant pattern match
Summary: as title

Reviewed By: haoxuany

Differential Revision: D13553851

fbshipit-source-id: 3bf8b89f76c7567776b19e5bcf62160b2b110984
2019-01-02 12:45:55 -08:00
Martin Heuschober
36eda02097 add support for setting the port (#281)
Summary:
+ based on https://stackoverflow.com/a/51671356/630160 I have implemented:
    - passing in `--port=8080`
    - or setting PORT=8080 as environment variable
  where  `default < PORT < --port`, i.e. a --port parameter overrides both an environment variable and a default value, and an environment variable PORT overrides the default value.

+ change small hlint: `fromMaybe def . fmap f = `maybe def f`
Pull Request resolved: https://github.com/facebook/duckling/pull/281

Differential Revision: D13388735

Pulled By: patapizza

fbshipit-source-id: cbc4a819036dbec474526b862b0f6187f02b6155
2018-12-12 11:15:52 -08:00
uugan
69ffb0f39d Mongolian (#277)
Summary:
MN Setup+Numeral
Pull Request resolved: https://github.com/facebook/duckling/pull/277

Reviewed By: chinmay87

Differential Revision: D13028169

Pulled By: patapizza

fbshipit-source-id: 7d587300918b3ed6211b6ce4147350147b4dc89e
2018-11-26 15:31:08 -08:00
Vitaly Shaydurov
b40e2147a9 Time/UK: added dimensions
Summary:
- added Time dimension for UK language
- added tests for Time dimension for UK language

Reviewed By: patapizza

Differential Revision: D13024050

fbshipit-source-id: e5be933fff8041c42d8fd26ddaf919584912a012
2018-11-19 04:31:03 -08:00
Akhil Ravidas
c46dbb0d92 Support malayalam numerals
Summary: Add support for malayalam numerals in duckling.

Reviewed By: patapizza

Differential Revision: D10075358

fbshipit-source-id: ce7fee2a71d862391980481b018c513981249f3d
2018-09-28 10:30:54 -07:00
kumar-shridhar
4a77765134 Icelandic support
Summary: Pull Request resolved: https://github.com/facebook/duckling/pull/247

Reviewed By: beauby

Differential Revision: D9656660

Pulled By: patapizza

fbshipit-source-id: 993baa9a0e442fc6f6f813a3af717b700c4d89e6
2018-09-06 06:46:02 -07:00
Gerben Janssen van Doorn
f70f991b38 Added support Lao numerals
Summary: Duckling didn't support Lao numerals, this diff adds it.

Reviewed By: patapizza

Differential Revision: D9323242

fbshipit-source-id: 5dad0d4dfb4843281a327947690e664c91ab8f1a
2018-08-17 10:31:17 -07:00
Julien Odent
f8c967c28d ExampleMain: show how to query for custom dimensions
Summary:
After including a custom dimension in `duckling.cabal` and importing
the constructor in `ExampleMain.hs`, you can query it using the `dims`
parameters (e.g. `dims=["my-dimension"]`).
Note that a custom dimension file should only expose the constructor (i.e.
`MyDimension(..)`) (and maybe a `corpus`).

https://github.com/facebook/duckling/issues/236

Reviewed By: adanielescu

Differential Revision: D9315608

fbshipit-source-id: 4908e7e27b31201c440a0c0db4cd3d567bd1e6f8
2018-08-16 15:30:43 -07:00
Ziyang Liu
a365dec52a Update custom dimension example to include pattern match on dimension data
Summary: Pull Request resolved: https://github.com/facebook/duckling/pull/233

Reviewed By: watashi

Differential Revision: D9021889

Pulled By: chinmay87

fbshipit-source-id: 1e134cf6d20006e1ee8824fa2fcbb8f7cdc4f197
2018-08-13 16:55:07 -07:00
PhalPheaktra Chhaya
b541354c31 Add Numeral dimension for new language KM. (#234)
Summary:
Hello,
I am new to Haskell, but I would like to add Khmer language (KM) to Duckling.
I have tried to extended Duckling by adding Numeral dimension for new language KM.
Please have a look at it and see what we can improve.

Thanks!
Pull Request resolved: https://github.com/facebook/duckling/pull/234

Reviewed By: blandinw

Differential Revision: D9032639

Pulled By: chinmay87

fbshipit-source-id: 7db19edf732fe6500629cc89e18e0655d7bbc48b
2018-08-03 14:45:55 -07:00
Arunavha Chanda
d5555d0149 Numeral/BN: Adding Bengali numeral support to Duckling
Summary: Added support for Bengali numerals

Reviewed By: patapizza

Differential Revision: D8730468

fbshipit-source-id: dc36017e24d796f35abc477a0b8b317218c64a6a
2018-07-09 12:30:30 -07:00
Julien Odent
f5bd1efb4a Time/EN: fix Palm Sunday regex
Summary:
* Fixes https://github.com/facebook/duckling/issues/215.
* `regenLangClassifiers` helper

Reviewed By: chinmay87

Differential Revision: D8663109

fbshipit-source-id: 5cbbc05729caabadeb0a88190ce00312174a4fdb
2018-06-27 12:30:42 -07:00
Tero Laxström
4ed1ed83ed Basics for Finnish (#210)
Summary:
Adds Locale and Numeral for Finnish
Closes https://github.com/facebook/duckling/pull/210

Reviewed By: JonCoens

Differential Revision: D8430386

Pulled By: patapizza

fbshipit-source-id: a3c8b3b3419b7f43e2ef332cdb1fb8fc07da3bec
2018-06-19 10:45:27 -07:00
Julien Odent
f272a175d4 Time/NL: Fix Sinterklaas for Belgium
Summary: In south Netherlands and in Belgium, Sinterklaas is celebrated on December 6.

Reviewed By: mullender

Differential Revision: D8281767

fbshipit-source-id: 4fc71783b9f543c4dfb04b8ba8481935cfb30dd8
2018-06-05 16:00:43 -07:00
Ziyang Liu
5460d8df0e Support custom dimensions
Summary:
Support custom dimensions

Had to move the definition of `Dimension` from `Duckling.Dimensions.Types` to `Duckling.Types` to avoid cyclic imports between these two modules.

A sample custom dimension is in `exe/CustomDimensionExample.hs`.

Limitations of custom dimensions:

- All rules for a custom dimension must be in the same module with the definition of the custom dimension. Otherwise there will be cyclic imports, because the definition of the dimension and the rules refer to each other.
- The custom dimension must be specified when using `parse`, since there's no way to get all the existing custom dimensions.

Reviewed By: patapizza

Differential Revision: D7630360

fbshipit-source-id: 30e12dcb33611f5692c4f5949de377bf61b75e1e
2018-04-19 15:30:51 -07:00
Giri Anantharaman
519c9519a3 Support Tamil numerals
Summary:
* Setup Tamil (TA) language
* Added Numeral Dimension

Reviewed By: patapizza

Differential Revision: D7323636

fbshipit-source-id: 4b1a42197ff4799880cded9ce86b8d7fae1507bc
2018-03-19 16:45:36 -07:00
Chinmay Deshmukh
5ac990bbe2 Return latent entities
Summary: Add an option to return latent time entities. This can be used when one is pretty certain that the input contains a datetime.

Reviewed By: patapizza

Differential Revision: D7254245

fbshipit-source-id: e9e0503cace2691804056fcebdc18fd9090fb181
2018-03-19 14:45:27 -07:00
bidhan-a
43079e7113 Setup Nepali (NE) and add Numeral dimension
Summary:
- Setup Nepali (NE) language
- Add basic Numeral dimension
Closes https://github.com/facebook/duckling/pull/156

Reviewed By: JonCoens

Differential Revision: D6965558

Pulled By: patapizza

fbshipit-source-id: f46c9b104d4345f20bd0cf53f8c9c8754855f314
2018-02-13 07:45:31 -08:00
Panagiotis Vekris
f28ad7a367 support for reference time param in web server
Summary:
PR for the issue #132
Closes https://github.com/facebook/duckling/pull/133

Reviewed By: patapizza

Differential Revision: D6661000

Pulled By: panagosg7

fbshipit-source-id: da061cb55e86c8747ff885df9881e270d22cdfa7
2018-01-10 12:45:37 -08:00
Abdallatif Sulaiman
1393098bcc Added Time Dimension to Arabic
Summary:
Hi, in this pr:
* Added time dimension to Arabic language, thanks to Hussein-Dahir & Yazeed-Obaid for writing time corpus.
* Fixed some bugs in numeral & ordinals and added more test cases for them.

Also, I don't really understand why do we use classifiers in time dimension?
Closes https://github.com/facebook/duckling/pull/123

Reviewed By: blandinw

Differential Revision: D6583313

Pulled By: patapizza

fbshipit-source-id: f7acdef0c032d7b7fd7d224832fdaf484d2df825
2017-12-19 14:30:42 -08:00
Newinfinite007
c133bad24a Hindi Language Numeral Dimension(minimalistic model). Tests passed.
Summary: Closes https://github.com/facebook/duckling/pull/119

Reviewed By: JonCoens

Differential Revision: D6597628

Pulled By: patapizza

fbshipit-source-id: 8bac0f686d6cecc38d9998e37042fe48f73530dc
2017-12-19 13:15:30 -08:00
Panagiotis Vekris
12a726aee7 Support for Greek times and dates
Summary:
This adds support for greek times.

There are still some issues with expressions of the form:
```
9:30 - 11:00 την πέμπτη
```
Where `11:00 την πέμπτη` is parsed first (as 11:30 on Thu), instead of prioritizing `9:30 - 11:00` as the training data suggests. These tests are for the moment excluded from the corpus.

Reviewed By: patapizza

Differential Revision: D6376271

fbshipit-source-id: 2f31e058fb88386429070e3b51cd33f93b9c5936
2017-12-04 16:45:40 -08:00
Panagiotis Vekris
fda8c7c759 Support Greek numerals
Summary:
- Setup Greek language (EL)
- Added Greek Numerals

Reviewed By: patapizza

Differential Revision: D6217873

fbshipit-source-id: 379170f
2017-11-02 17:16:18 -07:00
Matthijs Mullender
1ade1935b2 Support Dutch dates and times
Summary: [Duckling][Time][NL] Support Dutch dates and times

Reviewed By: patapizza

Differential Revision: D6090294

fbshipit-source-id: 54b8729
2017-10-19 14:04:38 -07:00
Julien Odent
1ab5f447d2 en_CA + fix Canadian Thanksgiving
Summary:
* `en_CA` locale
* In Canada, Thanksgiving Day is the second Monday of October.
* Black Friday is the same as the US.
* However Canada observes both DDMM and MMDD formats. Defer to later, falling back to US.

Reviewed By: blandinw

Differential Revision: D6058909

fbshipit-source-id: 3d4e05e
2017-10-16 10:04:43 -07:00
Julien Odent
fb1dcaa138 Chinese locales + fix TW National Day
Summary:
* Moving `ruleNationalDay` from `ZH` rules to specific locales: `zh_CN`, `zh_HK`, `zh_MO`
* Fixed National Day for `zh_TW`.

Reviewed By: blandinw

Differential Revision: D6057565

fbshipit-source-id: 8f9f2ab
2017-10-13 17:04:43 -07:00