phikoehn
344b150372
bug fixes with escaping / truecasing interactions
2013-01-14 19:22:29 +00:00
Mark Fishel
01eb79cf6f
fixed berkeley parser wrapper's broken output in case of in-text parentheses
2012-11-07 11:18:36 +01:00
Phil Williams
b2b9751227
parse-de-bitpar.perl: fix special char handling
...
Unescape special characters in input to BitPar and then re-escape
in output.
2012-10-06 16:27:33 +01:00
Hieu Hoang
debe090426
Change Bin to RealBin. Thanks to Tom Hoar
2012-06-26 11:57:23 -04:00
Hieu Hoang
0e621fe2d3
gzip grammar files
2012-06-08 00:43:47 +01:00
Hieu Hoang
82ca9fba69
delete example suffix array files
2012-06-07 19:40:10 +01:00
Hieu Hoang
03ce774d0e
suffix array creation script also creates glue grammar
2012-06-07 19:37:11 +01:00
Hieu Hoang
0b59bee21c
sa extraction works
2012-06-06 19:17:21 +01:00
Hieu Hoang
f0f4893a3d
start integrating adam's suffix array
2012-06-01 16:18:34 +01:00
Hieu Hoang
86666c2a1f
start integrating adam's suffix array
2012-06-01 16:06:37 +01:00
phikoehn
2e370ed11b
more escaping in tokenizer; wrapper for berkeley parser (german)
2012-05-30 00:58:18 +01:00
Hieu Hoang
1008b7c27b
Begin integrating Adam's suffix arrays into EMS
2012-05-14 19:44:53 +01:00
Hieu Hoang
93d0cc8313
Begin integrating Adam's suffix arrays into EMS
2012-05-14 18:56:10 +01:00
phikoehn
366d427ce6
minor fixes
2012-04-12 00:25:57 +01:00
phikoehn
791b5a7676
lotsa minor changes: mostly bug fixes, tokenizer now esacapes special Moses characters (|<>&)
2012-03-20 04:57:37 +00:00
Phil Williams
e70cafa1d7
The current version of BitPar's German grammar is encoded using UTF-8, so
...
disable the conversion to and from ISO-8859-1 in the parse-de-bitpar.perl
wrapper script. For compatibility with older versions of BitPar, the old
behaviour is available using the --old-bitpar command-line option.
2011-11-08 15:05:02 +00:00
hieuhoang1972
57e3a92836
rollback. argument not supported by all iconv
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3712 1f5c12ca-751b-0410-a591-d2e778427230
2010-11-15 12:50:11 +00:00
hieuhoang1972
ff339e56e3
don't drop unknown char. replace it with improbable string. avoid misalignment
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3709 1f5c12ca-751b-0410-a591-d2e778427230
2010-11-14 20:50:15 +00:00
hieuhoang1972
f7904a871c
add scripts to exclude unparseable sentences
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3704 1f5c12ca-751b-0410-a591-d2e778427230
2010-11-12 14:43:52 +00:00
hieuhoang1972
687cf9bf29
add scripts to exclude unparseable sentences
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3702 1f5c12ca-751b-0410-a591-d2e778427230
2010-11-12 14:20:11 +00:00
hieuhoang1972
a79a6bbaec
add scripts to exclude unparseable sentences
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3700 1f5c12ca-751b-0410-a591-d2e778427230
2010-11-11 18:04:16 +00:00
hieuhoang1972
f1f04daa0a
add empty line if input is empty line
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3699 1f5c12ca-751b-0410-a591-d2e778427230
2010-11-10 12:11:55 +00:00
phkoehn
85a5a13e4c
improvements to web analysis, fixes to syntax wrappers
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3633 1f5c12ca-751b-0410-a591-d2e778427230
2010-10-21 09:49:27 +00:00
hieuhoang1972
eedef63277
keep perl scripts with Unix line endings
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3612 1f5c12ca-751b-0410-a591-d2e778427230
2010-10-11 11:32:27 +00:00
phkoehn
b271862d7c
various updates, mostly related to experiment.perl
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3262 1f5c12ca-751b-0410-a591-d2e778427230
2010-05-18 17:39:16 +00:00
phkoehn
2ed6804f12
official release of experiment.perl
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3224 1f5c12ca-751b-0410-a591-d2e778427230
2010-05-04 23:04:10 +00:00
pjwilliams
2edfc16912
Merge remaining script support for tree-based models from mt3_chart.
...
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3137 1f5c12ca-751b-0410-a591-d2e778427230
2010-04-16 09:45:51 +00:00
