Commit Graph

1592 Commits

Author SHA1 Message Date
bhaddow
ee2ae991e5 Roll-back to non-reproducible, but transitive Compare operation
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2988 1f5c12ca-751b-0410-a591-d2e778427230
2010-03-19 16:59:08 +00:00
chardmeier
ac942f8358 Removed requirement that the number of probabilities cached in a
translation model for a given model be equal to the number of
feature functions provided by the model so collapseff lexical
reordering models work.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2921 1f5c12ca-751b-0410-a591-d2e778427230
2010-02-20 09:55:02 +00:00
sarst
435539d4ed Added an option to choose exactly which steps to run, not only the first and the last (since we sometimes need to do step 5 and 7 to retrain lexical reordering)
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2917 1f5c12ca-751b-0410-a591-d2e778427230
2010-02-19 18:03:44 +00:00
sarst
b95cc2f556 Added the check from word-based models of the alignment points in the adjacent corners, to the more complex models.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2916 1f5c12ca-751b-0410-a591-d2e778427230
2010-02-19 15:15:24 +00:00
sarst
bdfe47f99b added option to skip phrase-scoring, when we want to train new reordering models on old trainings
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2914 1f5c12ca-751b-0410-a591-d2e778427230
2010-02-18 21:16:00 +00:00
sarst
03e24c9856 changed the maximum model config, to always print all models
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2912 1f5c12ca-751b-0410-a591-d2e778427230
2010-02-18 13:45:15 +00:00
sarst
a86470c974 lexical reordering models: it is now possible to add the maximal reordering orientations to the extract file, and the collapsing information is no longer part of the filename. Also removed some old variables, that are not used anymore.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2911 1f5c12ca-751b-0410-a591-d2e778427230
2010-02-18 13:07:32 +00:00
chardmeier
b10d8eb469 Added lex reo scorer to released-files and removed extra ||| from reordering table.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2894 1f5c12ca-751b-0410-a591-d2e778427230
2010-02-12 16:23:32 +00:00
sarst
c65945b531 Cleaned up lescial reordering scoring, and sent vectors as references instead of copying them. Fixed bugs in extract: it used to choose the wrong orientation at end of sentences, and the hierarchical model typ is no longer dependent on the phrase-based model type.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2892 1f5c12ca-751b-0410-a591-d2e778427230
2010-02-12 13:46:33 +00:00
chardmeier
1b1f786373 Added missing namespace reference.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2891 1f5c12ca-751b-0410-a591-d2e778427230
2010-02-12 12:22:17 +00:00
chardmeier
7117fef52a Fixed dependency on previous scores for lexical reordering states.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2889 1f5c12ca-751b-0410-a591-d2e778427230
2010-02-11 17:08:24 +00:00
sarst
e812b2a8e3 Bugfixes: printing the correct phrases in the table, and fixed misspelling of monotonicity.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2885 1f5c12ca-751b-0410-a591-d2e778427230
2010-02-11 09:09:44 +00:00
sarst
92368ba490 Rewrote the lexical reordering model scoring in C++. Adapted train-factored-phrase-model.perl to that change. Minor fixes in other places, for compatibility
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2884 1f5c12ca-751b-0410-a591-d2e778427230
2010-02-10 17:19:06 +00:00
chardmeier
9adc3ee500 FFState ordering uses hypothesis ids now.
Changed order of scores to match latest trunk (exponential distortion next to lexical reordering).


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2844 1f5c12ca-751b-0410-a591-d2e778427230
2010-02-03 13:10:31 +00:00
naditomeh
242d6c6ddd word-based, phrase-based and hierarchical reordering is implemented in the training
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2823 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-31 23:56:45 +00:00
sarst
0785c09d03 bugfix for fatored-training config
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2804 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-30 09:34:55 +00:00
sarst
f2a5678541 added new file hierarchical.h
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2803 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-30 09:16:27 +00:00
sarst
6a13b8d186 bugfixes in train-factored-phrase-model*.perl
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2801 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-30 00:09:45 +00:00
chardmeier
be66463f5a Changed 'word' reordering type to 'wbe'. Added explanatory comments.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2800 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-29 23:58:07 +00:00
chardmeier
177277b7de Fixed score order in MSLR model.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2799 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-29 23:46:34 +00:00
chardmeier
1e076d51e7 Cleaned up forward reordering model.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2798 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-29 23:08:33 +00:00
chardmeier
2375365cf6 Fixed forward model.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2797 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-29 22:47:36 +00:00
sarst
bf70dd4767 subimtted working scripts for hierarchical training (msd)
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2796 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-29 22:38:18 +00:00
naditomeh
ad3b0760b2 adding extract.cpp
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2770 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-29 13:59:04 +00:00
naditomeh
03de8a99d8 adding extract.cpp
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2769 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-29 13:41:34 +00:00
sarst
4eec020d5b bugfixes to train-factored-phrase-model.perl
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2764 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-29 12:11:10 +00:00
sarst
766a0f95c0 allowing word in configuration of lexical reordering
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2761 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-29 11:37:12 +00:00
chardmeier
c73d90c911 Removed debugging instrumentation.
Changed score producer name to look more like the others.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2760 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-29 11:33:28 +00:00
sarst
a9ef19edf0 updated train-factored-phrase-model.perl to work with the new hierarchical reordering framework
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2759 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-29 11:29:39 +00:00
chardmeier
c8674d45c0 Added comparison to self in FFStateArray::Compare.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2756 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-29 11:05:41 +00:00
chardmeier
1d879ac349 Reenabled FFStateArray comparison.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2738 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-28 19:27:49 +00:00
chardmeier
8e43a5dca0 Fixed FFStateArray comparison.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2734 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-28 18:56:53 +00:00
chardmeier
96449ab552 Safer casting in LexicalReorderingState.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2729 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-28 18:26:48 +00:00
chardmeier
f49ddf97a4 Fixed some comparisons.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2722 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-28 16:43:58 +00:00
chardmeier
9b14482afb Bugfixes in the lexical reordering code.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2720 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-28 16:39:11 +00:00
sarst
4b40d26e0f updated quearyLexicalTable.cpp to fix compilation error
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2715 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-28 15:29:11 +00:00
chardmeier
42275ef916 Fixed segfault in LexicalReordering.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2713 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-28 15:13:11 +00:00
chardmeier
9c5a9b8dac Fixed bugs in LexicalReordering.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2711 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-28 14:55:20 +00:00
sarst
78c74e3c17 made contructor private in LexicalReorderingState
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2707 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-28 14:23:35 +00:00
chardmeier
74d27c02fa Added hierarchical reordering code to Makefile.am.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2706 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-28 14:01:20 +00:00
chardmeier
9c33fa2289 Hierarchical reordering stuff now compiles.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2699 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-28 12:12:57 +00:00
chardmeier
a3e9af21c3 Commented out unused variable.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2698 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-28 12:11:09 +00:00
chardmeier
38688cc30c LexicalReordering.cpp compiles now. Interface to StaticData still to be done.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2697 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-28 10:39:43 +00:00
sarst
621acd4dbf minor fixes for reordering
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2694 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-27 18:11:20 +00:00
sarst
6beccca7be minor fixes to the lexical reordering code
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2693 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-27 17:35:03 +00:00
sarst
24978ae2c8 added implementation of state for lexical reordering
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2692 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-27 17:10:35 +00:00
ankitks
ec917c460a added ReorderingStack class (Marathon 2010 project)
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2691 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-27 17:03:50 +00:00
sarst
8764077b39 nminor updates
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2689 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-27 14:34:49 +00:00
ankitks
d1eec53aac added ReorderingStack header file
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2688 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-27 14:30:23 +00:00
ankitks
980da78a71 added ReorderingStack header file
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/hierarchical-reo@2687 1f5c12ca-751b-0410-a591-d2e778427230
2010-01-27 14:25:01 +00:00