moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-11-13 10:40:52 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,404 Commits 136 Branches 26 Tags 147 MiB
bfbbefd710
Commit Graph

1404 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
phkoehn
bfbbefd710 bug fixes 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1917 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-26 03:32:29 +00:00
mphi
8a4c6a2c63 pus significance test into proper location 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1915 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-23 09:16:33 +00:00
mphi
88d3b775ce altered the bootstrap significance script algorithm according to (Riezler and Maxwell 2005 @ MTSE'05) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1914 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-23 09:03:41 +00:00
phkoehn
a09242ad16 bug fix with phrase table name in moses.ini, when using hmm alignment 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1913 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-20 21:37:57 +00:00
mphi
f033e32979 Added implementation of Koehn's 2004 EMNLP paired bootsrap resampling 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1911 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-20 11:55:12 +00:00
phkoehn
1c7b305152 bug fix 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1910 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-19 07:32:48 +00:00
phkoehn
1b5d99ad26 added headers for standard compliance (gcc 4.3 on 64 bit linux) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1905 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-16 21:14:38 +00:00
phkoehn
3a5981ce9d major improvements, see email to moses-support 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1904 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-15 23:25:14 +00:00
phkoehn
614876771d extended extract/score, to allow for one big file, not just parts 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1903 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-15 22:12:56 +00:00
hieuhoang1972
d9d1b8f748 fix caching bug. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1902 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-14 19:25:18 +00:00
hieuhoang1972
a88239c5c0 unix build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1901 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-10 18:31:07 +00:00
hieuhoang1972
254005bcee unix build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1900 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-10 18:30:59 +00:00
hieuhoang1972
6a61ffa9bd unix build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1899 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-10 18:30:51 +00:00
hieuhoang1972
868428fe66 unix build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1898 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-09 00:20:39 +00:00
hieuhoang1972
928d771085 create namespace 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1897 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-08 23:51:26 +00:00
redpony
76090cdda0 more normalization of feature names 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1896 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-05 19:34:49 +00:00
redpony
624e87a08f add more logging 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1895 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-05 18:12:42 +00:00
redpony
849538f73b make feature names the same. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1894 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-05 16:37:42 +00:00
hieuhoang1972
1ea1d4f9b1 always with unix line endings 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1893 1f5c12ca-751b-0410-a591-d2e778427230
 2008-09-29 19:34:54 +00:00
hieuhoang1972
68a2461cb3 vs build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1892 1f5c12ca-751b-0410-a591-d2e778427230
 2008-09-24 17:04:29 +00:00
redpony
a81390c5bb fix a couple names 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1891 1f5c12ca-751b-0410-a591-d2e778427230
 2008-09-24 17:03:07 +00:00
redpony
232dc9889c enable moses to accept a file that lists feature name and weight pairs. 
enable moses to export its search graph as a phrase lattice encoded serialized in a Google protocol buffer. This requires protoc (http://code.google.com/p/protobuf/) to function, disabled by default.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1890 1f5c12ca-751b-0410-a591-d2e778427230
 2008-09-24 16:48:23 +00:00
redpony
bb0ade93f7 a little refactoring in preparation for yet another way to export the search lattice. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1889 1f5c12ca-751b-0410-a591-d2e778427230
 2008-09-23 19:39:56 +00:00
hieuhoang1972
8ba603c39e visual studio build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1888 1f5c12ca-751b-0410-a591-d2e778427230
 2008-09-14 01:31:02 +00:00
nicolabertoldi
9cbde412e2 support for creating binary Phrase Tables including word-to-word alignments 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1887 1f5c12ca-751b-0410-a591-d2e778427230
 2008-09-12 18:19:41 +00:00
nicolabertoldi
dd6c36640b Support for printing out word-to-word alignments (besides phrase-to-phrase alignments) 
as contained in the phrase table.
If PT contains word-to-word alignments between source and target phrases,
Moses can optionally output them in the nbest and in the log file (if verbose).
W2w alignments from source to target and from target to source can differ,
if they differ in the PT.

Detailed documentation will be added in the Moses webpages very soon.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1886 1f5c12ca-751b-0410-a591-d2e778427230
 2008-09-12 18:09:06 +00:00
nicolabertoldi
e376f9f994 mv some Timer functions into the .cpp file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1885 1f5c12ca-751b-0410-a591-d2e778427230
 2008-09-12 15:31:46 +00:00
bhaddow
cd28f119c6 mert tests 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1881 1f5c12ca-751b-0410-a591-d2e778427230
 2008-09-01 17:18:51 +00:00
jdschroeder
ea5ddd4d82 fixed nasty out-of-bounds array read in WordsBitmap, simplified (fixed?) lattice checks. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1879 1f5c12ca-751b-0410-a591-d2e778427230
 2008-08-26 17:04:43 +00:00
jdschroeder
78534c1518 made all zcat calls through ZCAT variable. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1875 1f5c12ca-751b-0410-a591-d2e778427230
 2008-08-15 16:15:51 +00:00
mfederico
842f5842a2 Integrated handling of oov penalty into irstlm library. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1874 1f5c12ca-751b-0410-a591-d2e778427230
 2008-08-05 11:56:32 +00:00
mfederico
7a0e5811ba Fixed bug with dub option. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1873 1f5c12ca-751b-0410-a591-d2e778427230
 2008-08-05 11:44:50 +00:00
hieuhoang1972
5dee5d04aa rename IOStream to IOWrapper. 
move vs.net solution file to root folder

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1872 1f5c12ca-751b-0410-a591-d2e778427230
 2008-08-05 00:24:45 +00:00
maurocettolo
90e3107ef4 just commented a print on stderr 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1871 1f5c12ca-751b-0410-a591-d2e778427230
 2008-08-04 16:55:21 +00:00
mfederico
9e61900fad Fixed bug concerning the handling of the oov penalty with IRSTLM 
Now, the penalty for out-of-vocabulary words is specified 
by the parameter 

-lmodel-dub: dictionary upper bounds of language models

For instance, if you set it lmodel-dub to 1000000 (1M) and your actual 
vocabulary is let me say 200000 (200K), then the LM probabilty  of the
OOV word-class is divided by 800000 (800K), i.e. 1M-200K

You have to make sure that lmodel-dub is always larger than the LM 
dictionary.



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1870 1f5c12ca-751b-0410-a591-d2e778427230
 2008-08-04 13:06:52 +00:00
jdschroeder
7a2ebedc20 minor bugfixes and error checking 
-added -rootdir option to enhanced-mert
	-fixed float regex in score-nbest.py and mert-moses.pl
	-allow for extra weights in constructing ini in mert-moses.pl
	-additional NFS bug checks in mert-moses.pl



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1869 1f5c12ca-751b-0410-a591-d2e778427230
 2008-08-01 10:24:01 +00:00
hieuhoang1972
dd9691d28c abort if try to get substring of confusion network. returning emptyy string just screws things up 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1868 1f5c12ca-751b-0410-a591-d2e778427230
 2008-07-30 16:29:51 +00:00
hieuhoang1972
03bd63e312 get rid of md5 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1867 1f5c12ca-751b-0410-a591-d2e778427230
 2008-07-28 10:10:07 +00:00
bojar
6a087d59c4 removed SCRIPTS_ROOTDIR from this 'my' declaration, it was obscuring previous 
declaration!
lines wrapped


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1865 1f5c12ca-751b-0410-a591-d2e778427230
 2008-07-14 16:24:18 +00:00
bojar
2afe9e0357 avoid coredump files in parallel moses (usually just kills NFS for a while), 
debug on a smaller scale, if needed


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1864 1f5c12ca-751b-0410-a591-d2e778427230
 2008-07-14 13:58:42 +00:00
hieuhoang1972
620b0c34cc abort if internal LM asked to do more than trigram 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1863 1f5c12ca-751b-0410-a591-d2e778427230
 2008-07-11 14:55:06 +00:00
bojar
c20e682f18 Avoid NFS race condition: 
explicitly remove old cmert output files (hoping that they will be correctly
  replaced by a 'mv' in the shell script submitted to SGE by qsubwrapper
  occasionally reveals a race condition in NFS => weights seem unchanged =>
  mert finishes too early)


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1862 1f5c12ca-751b-0410-a591-d2e778427230
 2008-07-10 11:47:55 +00:00
saintamh
9d106392e6 Bugfix 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1861 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-30 09:18:14 +00:00
bhaddow
83f234cf17 Implementation of Cer et al mert regularisation. Use with argument such 
as --scconfig regtype:min,regwin:3 in extractor and mert. Only tested
on toy example so far.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1860 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-24 19:27:18 +00:00
hieuhoang1972
6ddde13dca fixed constraint format bug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1859 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-20 20:24:16 +00:00
dowobeha
3e1c6c39ff Fixed constraint decoding - there may be a bug in Util::Tokenize 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1858 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-20 15:45:06 +00:00
hieuhoang1972
81c7e5118b must provide line no to constraint file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1857 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-19 22:47:54 +00:00
dowobeha
67c8bdd328 Constraint decoding works, but not for cube pruning. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1856 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-19 22:40:51 +00:00
hieuhoang1972
52c2843e6c perl regexpr bug, submitted by German Sanchis Trilles 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1855 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-19 21:57:29 +00:00
dowobeha
7a4b1fb699 Added preliminary code for constraint decoding 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1854 1f5c12ca-751b-0410-a591-d2e778427230
 2008-06-18 23:14:09 +00:00