moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-12-29 06:52:34 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,674 Commits 136 Branches 26 Tags 147 MiB
795224736b
Commit Graph

1674 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
bhaddow
795224736b Merge revisions 2670-2988 from track. Passes all regression except lexicalised 
reordering


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2989 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-19 17:52:51 +00:00
bhaddow
ee2ae991e5 Roll-back to non-reproducible, but transitive Compare operation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2988 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-19 16:59:08 +00:00
bgottesman
5a3a6bd3b0 set utf8 mode on the input and output files, instead of on stdin and stdout, which are not used. This allows case variants of non-ASCII characters to be recognized correctly 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2987 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-18 19:13:05 +00:00
bhaddow
22b74b56f7 Output of nbest from lattice mbr 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2980 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-16 11:34:50 +00:00
bhaddow
0208bddb70 Added parameters for lmbr n-best 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2971 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-14 20:23:17 +00:00
bgottesman
872d50dd9b Fix a line that extracts the source FACTORLIST from a FACTORMAP (in the terms of the Moses manual) so that it succeeds even if the target FACTORLIST contains commas, i.e. contains multiple factors 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2965 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-10 18:03:53 +00:00
hieuhoang1972
1a4646793a put back in 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2961 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-08 20:24:37 +00:00
abarun
154b01a2ef More efficient implementation of print all derivations 2 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2958 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-08 16:30:31 +00:00
abarun
e09be2bfd5 More efficient implementation of print all derivations 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2957 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-08 15:28:40 +00:00
hieuhoang1972
8eeb1708ae minor 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2956 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-07 19:41:29 +00:00
hieuhoang1972
cbb329b754 minor 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2955 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-07 19:39:07 +00:00
abarun
86ef292af1 Writing out sentence number when printing all derivations 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2954 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-07 08:28:16 +00:00
abarun
3434d15cda Lattice MBR pruning based on target length of MAP hypothesis 
Added option (-disable-discarding) to disable discarding of hypos during search
Added option (-print-all-derivations) to output (in unsorted order) ALL derivations found in search graph 


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2953 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-07 07:57:48 +00:00
jfouet
79a8f0f133 restore trace by default 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2950 1f5c12ca-751b-0410-a591-d2e778427230
 2010-03-01 11:32:15 +00:00
jfouet
9501708565 use old macro name to enable LIBTOOL 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2947 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-25 15:36:39 +00:00
bojar
d7335c0de1 avoid reducing corpus file if it already contains the needed factors 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2945 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-24 16:14:07 +00:00
jfouet
42d1a8ab93 Disable shared library by default, use --enable-shared=yes to create libmoses.so 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2943 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-24 14:02:24 +00:00
bhaddow
ef54ec3699 Fix mosesmt compile 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2942 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-24 13:30:40 +00:00
jfouet
242846f07c Use portable include guard instead of pragma once 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2939 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-24 11:15:44 +00:00
jfouet
e14dc33dc8 enable trace with a configure options and fix compilation error when trace are disabled 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2938 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-24 10:37:49 +00:00
jfouet
c9af4f642d Add configurable path to zlib 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2937 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-24 10:35:18 +00:00
jfouet
087b2021c5 Link libmoses with boost-thread 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2936 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-24 10:34:00 +00:00
jfouet
5403d34faf pragma only valid with visual studio 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2935 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-24 10:30:24 +00:00
jfouet
133f18bd90 Use stringstream operation instead of atoi and atof to prevent error and overflow 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2934 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-24 10:03:22 +00:00
jfouet
50c22f1e42 Fix seg fault when libmoses is used ad a dynamic library on windows, MagicWord was initialized to a wrong value 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2933 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-24 10:02:17 +00:00
jfouet
7c1e287c71 Add configure options to link dynamically with libmoses 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2932 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-24 09:27:18 +00:00
jfouet
53c56df23e Add configure option to link with libsrilm.so 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2931 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-24 09:26:58 +00:00
jfouet
b72e7fa8d9 Add configure option to link with libsrilm.so 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2930 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-24 09:19:37 +00:00
bhaddow
0ade2ac432 Fix lattice mbr grid search so that the full lattice is created and pruned correctly. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2929 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-22 23:42:35 +00:00
bhaddow
0e16a9c3c5 deubg to cerr not cout 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2928 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-22 19:20:28 +00:00
chardmeier
ac942f8358 Removed requirement that the number of probabilities cached in a 
translation model for a given model be equal to the number of
feature functions provided by the model so collapseff lexical
reordering models work.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2921 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-20 09:55:02 +00:00
sarst
435539d4ed Added an option to choose exactly which steps to run, not only the first and the last (since we sometimes need to do step 5 and 7 to retrain lexical reordering) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2917 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-19 18:03:44 +00:00
sarst
b95cc2f556 Added the check from word-based models of the alignment points in the adjacent corners, to the more complex models. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2916 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-19 15:15:24 +00:00
sarst
bdfe47f99b added option to skip phrase-scoring, when we want to train new reordering models on old trainings 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2914 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-18 21:16:00 +00:00
bhaddow
b2201e765d Grid search for lattice mbr 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2913 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-18 14:15:34 +00:00
sarst
03e24c9856 changed the maximum model config, to always print all models 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2912 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-18 13:45:15 +00:00
sarst
a86470c974 lexical reordering models: it is now possible to add the maximal reordering orientations to the extract file, and the collapsing information is no longer part of the filename. Also removed some old variables, that are not used anymore. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2911 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-18 13:07:32 +00:00
hieuhoang1972
ad3b032ebf roll back 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2910 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-18 11:00:40 +00:00
bojar
cce14523ab added -report-all-factors-in-n-best 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2909 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-18 10:54:33 +00:00
bhaddow
7ebfb57818 Preallocate vector to speedup copy 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2908 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-17 19:05:34 +00:00
bhaddow
11cb44ba5b Refactoring of lattice mbr code so that it can be used with mosesmt 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2907 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-17 17:25:56 +00:00
bhaddow
7f45cd12f9 reduce memory usage in lattice mbr 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2906 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-17 14:14:44 +00:00
bhaddow
b3a8411ae1 Fix compile error 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2901 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-15 19:27:17 +00:00
hieuhoang1972
6b1b505089 visual studio 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2897 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-14 00:26:34 +00:00
abarun
6aff4f1afd Floor probability for unknown ngrams in Lattice MBR 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2896 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-12 18:04:17 +00:00
leven101
b63de914a5 added ability to go src->trg and trg->src phrase extraction in suffix array phrase implementation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2895 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-12 17:29:15 +00:00
chardmeier
b10d8eb469 Added lex reo scorer to released-files and removed extra ||| from reordering table. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2894 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-12 16:23:32 +00:00
abarun
0b04e54234 Only outputting surface form for LatticeMBR argmax 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2893 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-12 15:56:37 +00:00
sarst
c65945b531 Cleaned up lescial reordering scoring, and sent vectors as references instead of copying them. Fixed bugs in extract: it used to choose the wrong orientation at end of sentences, and the hierarchical model typ is no longer dependent on the phrase-based model type. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2892 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-12 13:46:33 +00:00
chardmeier
1b1f786373 Added missing namespace reference. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2891 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-12 12:22:17 +00:00