Summary:
Whle looking into fixing https://github.com/facebook/duckling/issues/380
I was having a bit of trouble navigating the existing rules and guessing
what is / is not supported.
This diff refactors the Numeral/ES code to be easier to navigate:
- rename all the `ruleNumeral{1,2,3,4,5,6}` rules to be descriptive
- changes the order to be themed from small to large numbers, and
make sure the order of defines matches the order of rules at the end
of the module
- use [20 .. 90] instead of manually specifying the same list out-of-order
Reviewed By: chessai
Differential Revision: D27858134
fbshipit-source-id: b13983d75b36bb4e2b387ef06fe61066d81ae19a
Summary:
It's common to use dashes when spelling out times longhand,
e.g. "five-thirty am", but Duckling wasn't handling this at all.
This commit adds rules for times spelled out with dashes. The
rules explicitly forbid the second of the two times from including
digits via a negative match. This is because
- it wouldn't be at all idomatic to write five-26 or five-oh-6
- allowing that pattern clashes with time range parsing, e.g.
"9-10 am" should parse as a time range, not as "9:10 am"
Reviewed By: chessai
Differential Revision: D27848428
fbshipit-source-id: dfe8b98cb38119a16db2a19db47fd3128783e617
Summary:
This commit fixes#111, which was an open issue that any non-integer
multiple of any unit of time was being converted to seconds.
My solution is to write a recursive function `Duration.Helpers.inCoarsestGrain`
which, given a grain `g` and double value `v` finds the coarses grain `g'` such that
`v * g` - rounded to the nearest seconds - has integral units.
We call this function only in the case of non-integer multiples, and we start our
search from the given grain because nothing coarser would make sense. The code could
actually be slightly more efficient if we started at the next-smallest grain, but
in the interest of clarity I think this is probably better.
Reviewed By: chessai
Differential Revision: D27891439
fbshipit-source-id: b048310963eb71337fd91ab4ef3c840134a76e73
Summary:
While debugging an attempt to extend our handling of spelled-out
times, I realized that we are being too aggressive in our parsing of
times like "five ten", because we'll parse "five nine" as possibly
meaning "5:09", which isn't something an English speaker would say
(or rather if they did, it's more likely they mean "five (to) six"
or something similar.
Reviewed By: chessai
Differential Revision: D27848429
fbshipit-source-id: 34d783332fd60359ad9b6e7862367453bc93a1d1
Summary:
This commit gets rid of all the easy-to-fix lint warnings on time helper modules:
- replacing unnecessary `.` with `$`
- Flipping a lambda in a map to an infix operation
- Use `ts` for a list of times, not `series` which produces a pretty confusing naming collision
There are still quite a lot of lint errors related to name masking, which would be challenging to fix without us coming to an agreement about naming conventions.
But at least in my editor, name-masking errors are a lot less visually noisy than other errors (they only highlight the one name) so I don't mind them as much when skimming the code.
Reviewed By: chessai
Differential Revision: D27842198
fbshipit-source-id: 9091e5349657243b61d7ee169d0d06dd2122ac17
Summary:
The Dutch numeral and ordinal rules contained a spelling mistake: [‘achttien’](https://en.wiktionary.org/wiki/achttien) (18) was spelt as ‘achtien’ (note the single *t*).
This PR fixes this spelling mistake everywhere in the code base.
Pull Request resolved: https://github.com/facebook/duckling/pull/602
Reviewed By: patapizza
Differential Revision: D27859001
Pulled By: chessai
fbshipit-source-id: 8da0d8b099d49cf6207d4066cee1fc7da68a418e
Summary:
While looking at bugfixes for some ES and IT numeral parsing, I was
looking at the Helpers.hs module and figured I'd silence some lint
errors. There are still some name-masking lint issues, but all others
have been fixed.
Reviewed By: chessai
Differential Revision: D27854164
fbshipit-source-id: 5be8d924b033a55608c455074df1c80c8c0019be
Summary:
This fixes#592 in a very conservative way: the reason why `ruleIntersect` does
not detect "tonight 815" and "tonight eight fifteen" as it does "tonight 8:15"
is because it explicitly forbids the second part of the intersection from being
latent, unless it is a year.
I don't think it's a good idea to remove the restriction on latent inputs in
`ruleIntersect`, so instead I just made a new rule specifically for the
intersection of `<part-of-day> <time-of-day>`.
It also seems to me that there's a lot of room for this to be too aggressive,
for example if I say "tonight 500 people will laugh" the "tonight" and "500"
aren't really linked. So, I set the rule to be latent; this may be too conservative
to be useful though (do client libraries usually allow latent results?).
Reviewed By: chessai
Differential Revision: D27842596
fbshipit-source-id: 36ac59e31c632d4864241bce291147a46d52f780
Summary: Adds Catalan language and Numeral rules for it
Reviewed By: haoxuany
Differential Revision: D26518604
Pulled By: chessai
fbshipit-source-id: e6b4b0ceb9b7931d086c732dd03fb5cbbe062d5b
Summary:
These are popular variants/abbreviations of Egyptian pounds.
All these forms are documented on wikipedia (https://en.wikipedia.org/wiki/Egyptian_pound)
Pull Request resolved: https://github.com/facebook/duckling/pull/590
Reviewed By: haoxuany
Differential Revision: D27598249
Pulled By: chessai
fbshipit-source-id: 42ae9115b1def48c58e50a6deb624c3407c029f3
Summary:
Convert `This` to `Seal` in order to make this example working.
Pull Request resolved: https://github.com/facebook/duckling/pull/585
Reviewed By: girifb
Differential Revision: D27235685
Pulled By: chessai
fbshipit-source-id: 71a712a622b5d9d10f7842276a2b8f60f962477e
Summary:
By default, Facebook Haskell code has a lint error banning
-1s, because at one point it was used as a default value in some
API handlers and it's better to use Option + Nothing for this use case.
But in Duckling - particularly in the Time module - it's quite
common that we actually want to work with -1 as a meaningful integer
value, involving some kind of offset.
Reviewed By: chessai
Differential Revision: D27118984
fbshipit-source-id: 5fe2200e8005a20855d7fdd3a8eb2ad33291edc8
Summary:
The facebook internal linters prefer us to avoid
excessive point-free style and extra $ where we could
instead move existing brackets.
Making those style tweaks for Time/EN/Rules.hs because
I was looking at the file as part of
Reviewed By: chessai
Differential Revision: D27108042
fbshipit-source-id: 7c8e76578476ea14d655131943e693c5159b12d2
Summary:
On some linux systems, such as on NixOS, /usr/share/zoneinfo does not
exist. What does exist in its place is /etc/zoneinfo. So, we should try to load
that if /usr/share/zoneinfo does not exist.
Pull Request resolved: https://github.com/facebook/duckling/pull/582
Reviewed By: girifb
Differential Revision: D27086925
Pulled By: chessai
fbshipit-source-id: f4a38822be9888d57034f67a6f7abd17d56d38b8
Summary:
I was testing an unrelated change (which doesn't change
classifier scores) and reran classifiers just to be safe, I noticed
that the scores changed.
This diff updates them.
Reviewed By: chessai
Differential Revision: D26892970
fbshipit-source-id: c7da3e3b7d01955f98b287a3ff4e7c1ff2837c7f
Summary:
I was looking at adding support for "next week" constructions in Spanish to
close https://github.com/facebook/duckling/issues/553 (which it appears has
already been handled), when I noticed that the equivalent logic for English
has been split into two separate examples: "coming week" isn't in the same
example as other equivalent constructs like "upcoming week" and "next week".
This diff combines them, which I think is clearer and fewer lines of code
Reviewed By: chessai
Differential Revision: D26892322
fbshipit-source-id: 68ca4644759198fc79d963ae080495c3f2d4a923
Summary: due to exploit in T85548324, factoring the input to get a smaller parse tree (the existing one parses tail recursively, whereas this one uses ruleIntersect which is still bad, but slightly better).
Differential Revision: D26657170
fbshipit-source-id: fe3a738073b4d30ae401521bb692f4a4bba48d96
Summary: There are a handful of more spelling for russian numbers [20, 30 .. 90] that we aren't handling. Additionally, we optimise for recall over precision by allowing some invalid spellings that could be understandable typos.
Reviewed By: patapizza
Differential Revision: D26285711
Pulled By: chessai
fbshipit-source-id: fd8a8f373d228a526e79b22326eff48bb966310d
Summary:
Add rules:
- `hkd` as HKD, and related rules (prefix and suffix)
- dollar and <amount-of-money> rule
- dollar and a half rule
- intersection for <amount-of-money> and `a half`
Changed:
- dime and dollar rules now have improved coverage
Reviewed By: girifb
Differential Revision: D26191724
Pulled By: chessai
fbshipit-source-id: bf63b6eaa751fb96dcf341fa2b66db06a6eeca79
Summary: Results in no change on linux/macos, but this is necessary on windows to prevent paths from being botched
Reviewed By: girifb
Differential Revision: D25893201
fbshipit-source-id: ca79dd8a766aecf27562044865d9bc258a4e8d11
Summary:
Adding . in between kilogram units used to be extracted as a Numeral
instead of Quantity.
Pull Request resolved: https://github.com/facebook/duckling/pull/570
Reviewed By: patapizza
Differential Revision: D26199687
Pulled By: chessai
fbshipit-source-id: 65e39f20296946d5762d7180b12878f4e66ea701
Summary:
In general there are some clashes between time formats `hhmm` and date formats `ddmm`. For example, depending on context, `22.10` can mean clock time ten past ten or the twenty second of october. In general it's correct to interpret this as clock time, as Duckling currently does.
But there are some cases not currently covered by Duckling where we have more unambiguous dates, e.g. `12.03.2018` and `27.11`. These are included here (in addition to midnight `24:00` which was also missing).
#### Changes:
- Bug in `ruleDdmm` regex meant that dates on the format `dd/mm` where `mm > 9` were not parsed
- `ruleYyyymmdd` now also parses dots and forward slashes, i.e. `2012.05.14` and `2012/05/14`
- New rule `rule2400` parses `24:00` and `24.00` (I elected not to include it in `ruleMidnighteodendOfDay` as it has grain minute rather than day)
- New rule `ruleDmm` parses `1/10`, `9.12` etc
- New rule `ruleDDm` parses `10/3`, `11.1` etc
- New rule `ruleDdDotMm` parses `25.02`, `31.10` etc
- `ruleDdmmyyyy` now also parses dots, i.e. `03.10.1983`
- New tests
Pull Request resolved: https://github.com/facebook/duckling/pull/395
Reviewed By: patapizza
Differential Revision: D26193069
Pulled By: chessai
fbshipit-source-id: cf711807fa1d40be2303f2426d74ded40c2e23b3
Summary:
This PR adds UAH currency Type and examples to EN and RU Corpus
Pull Request resolved: https://github.com/facebook/duckling/pull/433
Reviewed By: girifb
Differential Revision: D25102990
Pulled By: chessai
fbshipit-source-id: ed40e8dfcf145a65c7e6d87158da0efacb32e256
