Language, engine, and tooling for expressing, testing, and evaluating composable language rules on input strings.
Go to file
Julien Odent bfb6ba0387 Numeral flag for Time patterns
Summary:
Today things like `at single`, `at a few`, `at a couple of` would return a `Time`.
Discussed with blandinw to do this very explicit hack right now until other use cases show up.

Reviewed By: niteria

Differential Revision: D5325369

fbshipit-source-id: aec0402
2017-06-27 07:34:21 -07:00
Duckling Numeral flag for Time patterns 2017-06-27 07:34:21 -07:00
exe Add example modules for benchmarking and profiling 2017-06-22 09:19:18 -07:00
tests change json response #12 2017-06-12 15:19:22 -07:00
.gitignore Setup logs 2017-06-14 02:04:26 -07:00
.travis.yml Don't double build on travis 2017-06-22 06:34:53 -07:00
CONTRIBUTING.md CONTRIBUTING.md 2017-03-10 14:49:18 -08:00
Dockerfile Adding dockerfile for the http server 2017-05-17 10:19:44 -07:00
duckling.cabal Add example modules for benchmarking and profiling 2017-06-22 09:19:18 -07:00
LICENSE Initial commit 2017-03-08 10:33:56 -08:00
logo.png Adding logo 2017-03-15 08:04:31 -07:00
PATENTS Initial commit 2017-03-08 10:33:56 -08:00
README.md Steps for using Duckling.Debug 2017-05-11 07:34:22 -07:00
stack.yaml Upgrade to stackage 8.8 2017-04-05 11:19:31 -07:00

Duckling Logo

Duckling Build Status

Duckling is a Haskell library that parses text into structured data.

"the first Tuesday of October"
=> {"value":"2017-10-03T00:00:00.000-07:00","grain":"day"}

Requirements

A Haskell environment is required. We recommend using stack.

On macOS you'll need to install PCRE development headers. The easiest way to do that is with Homebrew:

brew install pcre

If that doesn't help, try running brew doctor and fix the issues it finds.

Quickstart

To compile and run the binary:

$ stack build
$ stack exec duckling-example-exe

The first time you run it, it will download all required packages.

This runs a basic HTTP server. Example request:

$ curl -XPOST http://0.0.0.0:8000/parse --data 'text=tomorrow at eight'

See exe/ExampleMain.hs for an example on how to integrate Duckling in your project.

Supported dimensions

Duckling supports many languages, but most don't support all dimensions yet (we need your help!).

Dimension Example input Example value output
AmountOfMoney "42€" {"value":42,"type":"value","unit":"EUR"}
Distance "6 miles" {"value":6,"type":"value","unit":"mile"}
Duration "3 mins" {"value":3,"minute":3,"unit":"minute","normalized":{"value":180,"unit":"second"}}
Email "duckling-team@fb.com" {"value":"duckling-team@fb.com"}
Numeral "eighty eight" {"value":88,"type":"value"}
Ordinal "33rd" {"value":33,"type":"value"}
PhoneNumber "+1 (650) 123-4567" {"value":"(+1) 6501234567"}
Quantity "3 cups of sugar" {"value":3,"type":"value","product":"sugar","unit":"cup"}
Temperature "80F" {"value":80,"type":"value","unit":"fahrenheit"}
Time "today at 9am" {"values":[{"value":"2016-12-14T09:00:00.000-08:00","grain":"hour","type":"value"}],"value":"2016-12-14T09:00:00.000-08:00","grain":"hour","type":"value"}
Url "https://api.wit.ai/message?q=hi" {"value":"https://api.wit.ai/message?q=hi","domain":"api.wit.ai"}
Volume "4 gallons" {"value":4,"type":"value","unit":"gallon"}

Extending Duckling

To regenerate the classifiers and run the test suite:

$ stack build :duckling-regen-exe && stack exec duckling-regen-exe && stack test

It's important to regenerate the classifiers after updating the code and before running the test suite.

To extend Duckling's support for a dimension in a given language, typically 2 files need to be updated:

  • Duckling/<dimension>/<language>/Rules.hs
  • Duckling/<dimension>/<language>/Corpus.hs

Rules have a name, a pattern and a production. Patterns are used to perform character-level matching (regexes on input) and concept-level matching (predicates on tokens). Productions are arbitrary functions that take a list of tokens and return a new token.

The corpus (resp. negative corpus) is a list of examples that should (resp. shouldn't) parse. The reference time for the corpus is Tuesday Feb 12, 2013 at 4:30am.

Duckling.Debug provides a few debugging tools:

$ stack repl --no-load
> :l Duckling.Debug
> debug EN "in two minutes" [This Time]
in|within|after <duration> (in two minutes)
-- regex (in)
-- <integer> <unit-of-duration> (two minutes)
-- -- integer (0..19) (two)
-- -- -- regex (two)
-- -- minute (grain) (minutes)
-- -- -- regex (minutes)
[Entity {dim = "time", body = "in two minutes", value = "{\"values\":[{\"value\":\"2013-02-12T04:32:00.000-02:00\",\"grain\":\"second\",\"type\":\"value\"}],\"value\":\"2013-02-12T04:32:00.000-02:00\",\"grain\":\"second\",\"type\":\"value\"}", start = 0, end = 14}]

License

Duckling is BSD-licensed. We also provide an additional patent grant.