moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-12-27 05:55:02 +03:00
Code Issues Packages Projects Releases Wiki Activity
11,112 Commits 136 Branches 26 Tags 147 MiB
4cc4913aa6
Commit Graph

3077 Commits

Author SHA1 Message Date
Hieu Hoang
68974c9f20 check there are hypos in stack vector before create translation options. constrained decoding works for syntax model 2013-09-18 04:56:57 +02:00
Hieu Hoang
7873dbb77a check there are hypos in stack vector before create translation options. constrained decoding works for syntax model 2013-09-18 04:45:19 +02:00
Hieu Hoang
f88908f972 add option to store rule source in target phrase 2013-09-17 18:29:06 +02:00
Hieu Hoang
3271d86c25 add option to store rule source in target phrase 2013-09-17 18:22:00 +02:00
Hieu Hoang
6c739bcbf5 constrained decoding FF works for both pb and hiero 2013-09-17 15:26:25 +02:00
Hieu Hoang
5ebb81a17a constrained decoding FF works for both pb and hiero 2013-09-17 15:06:17 +02:00
Hieu Hoang
764684bb6f Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-09-17 09:56:38 +02:00
Hieu Hoang
2e5246844f comment out deletion of hypos with -inf scores. Apparently, all stacks must have hypos, otherwise terrible things happens. This is weird, empty stacks is perfectly reasonable. Need to ask Phil about this 2013-09-17 06:07:42 +02:00
Hieu Hoang
8fe087a92d comment out deletion of hypos with -inf scores. Apparently, all stacks must have hypos, otherwise terrible things happens. This is weird, empty stacks is perfectly reasonable. Need to ask Phil about this 2013-09-17 03:43:04 +02:00
Hieu Hoang
21319acb88 implement ConstrainedDecoding FF 2013-09-17 03:13:32 +02:00
Hieu Hoang
8b2bed458c implement ConstrainedDecoding FF 2013-09-17 02:40:25 +02:00
Hieu Hoang
ef2f6167e7 implement ConstrainedDecoding FF 2013-09-16 20:45:43 +02:00
Hieu Hoang
3b13068ce4 oops. forgot to check in new FF 2013-09-16 11:00:11 +01:00
Hieu Hoang
0d94d7067e update convert-moses-ini-to-v2.perl with memory pt for SCFFG decoding. Start on constrained ff 2013-09-16 07:46:45 +02:00
Barry Haddow
1a8a8fbb2d Re-enable unit tests. They all pass. 2013-09-14 17:15:41 +01:00
maria nadejde
5f37a545b1 fixed sparse feature output 2013-09-14 10:44:35 +02:00
maria nadejde
bf5c32df6c stuff that probably doesn't work 2013-09-13 19:43:04 +02:00
maria nadejde
92d0932c27 Stateless Feature InternalStructStatelessFF 2013-09-13 18:23:24 +02:00
maria nadejde
7077a04c3b Stateless Feature InternalStructStatelessFF 2013-09-13 18:21:23 +02:00
maria nadejde
3c441bdaa3 Merge branch 'GHKMStruct' of github.com:moses-smt/mosesdecoder into GHKMStruct 2013-09-13 18:18:52 +02:00
maria nadejde
48fa62838c Merge branch 'master' into GHKMStruct 2013-09-13 17:30:50 +02:00
Matthias Huck
c39bed60c0 Tree fragments in GHKM glue rules; 
output of LHS tag in tree fragments for UNKs;
GHKMParse info is now denoted as Tree info
 2013-09-13 17:10:21 +02:00
Hieu Hoang
3b03d803d9 make sure Evaluate(inputPath) is called for chart translation options 2013-09-13 09:27:48 +02:00
Hieu Hoang
9950279bc7 add pointer to input path to chart translation option 2013-09-12 19:50:09 +02:00
Matthias Huck
225f33e325 tiny changes to GHKMParse info in search 2013-09-12 17:37:54 +02:00
Phil Williams
f3320c1ca0 Add -Tghkm parameter 
Writes internal tree structure (if available) for 1-best derivation.
Code contributed by Nina Seemann.
 2013-09-12 11:31:12 +02:00
Hieu Hoang
b534e8c4b7 f*cking link error 2013-09-12 09:17:18 +01:00
Hieu Hoang
c36d8d048f ExternalFeature /cdec & phil blumson 2013-09-12 09:48:17 +02:00
Hieu Hoang
f379e5cb8a lattice decoding with sparse features 2013-09-10 15:36:21 +02:00
Hieu Hoang
16cfc786a8 lattice decoding with sparse features 2013-09-08 19:22:55 +02:00
Hieu Hoang
42fc86b259 lattice decoding with sparse features 2013-09-08 17:02:59 +02:00
Hieu Hoang
7214be82d4 lattice decoding with sparse features 2013-09-08 16:43:18 +02:00
Hieu Hoang
1adf7d0521 lattice decoding with sparse features 2013-09-08 15:57:31 +02:00
Hieu Hoang
65e2806b1d move lattice scoring into InputFeature class 2013-09-08 13:47:32 +02:00
Kenneth Heafield
b27d140619 Unfortunately UTrieNode wants a non-explicit copy constructor 2013-09-07 09:51:09 +01:00
Lane Schwartz
7e1cfc981f Merge branch 'master' of www:/repos/git/Decoders/mosesdecoder 2013-09-06 14:01:32 -04:00
Lane Schwartz
226031ecf6 Checking in examples of CMake files. 
The example in contrib works, as does the one in biconcor.

The examples in lm and moses don't work at the moment,
but checking them in anyway as a starting point
for any potential work on them next week during the
MT Marathon.
 2013-09-06 13:39:46 -04:00
Ulrich Germann
384650fb60 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-09-06 17:05:39 +01:00
Ulrich Germann
64cfd3ec16 Minor cosmetic changes. 2013-09-06 17:04:46 +01:00
Ulrich Germann
8154654e7c Added using namespace std; 2013-09-06 17:02:32 +01:00
Hieu Hoang
8ae1614be4 no change commit. Force rerun of cruise control 2013-09-06 08:53:17 +02:00
Hieu Hoang
fbb2f9abb0 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-09-05 23:34:39 +02:00
Hieu Hoang
12c4a0a68e lattice decoding with sparse features 2013-09-05 23:34:22 +02:00
Hieu Hoang
6f81004e20 lattice decoding with sparse features 2013-09-05 22:26:19 +02:00
Hieu Hoang
8803bcdba6 lattice decoding with sparse features 2013-09-05 22:25:32 +02:00
Hieu Hoang
56c4a41231 lattice decoding with sparse features 2013-09-05 22:22:53 +02:00
Hieu Hoang
cec177b493 no change commit. Force rerun of cruise control 2013-09-05 18:31:45 +01:00
Hieu Hoang
9894941294 properties is on 7th column, not 6th 2013-09-03 15:02:53 +02:00
Hieu Hoang
01b7f8a1e8 add skeleton feature functions 2013-09-01 16:58:23 +01:00
Hieu Hoang
af6b1f2b4e start on forced decoding for SCFG 2013-08-31 01:37:15 +01:00
Hieu Hoang
769f2fe00c start on forced decoding for SCFG 2013-08-30 17:45:56 +01:00
Hieu Hoang
fe36cbf737 move LexicalReordering to FF directory 2013-08-30 16:28:28 +01:00
Hieu Hoang
04783309ba move DecodeFeature to FF directory 2013-08-30 16:03:06 +01:00
Hieu Hoang
03fd06b38b convert Evaluate() methods into abstract, rather than empty functions. Every FF must implement them all 2013-08-30 15:49:00 +01:00
Lane Schwartz
c0fe9bb4ea Updating XCode project 2013-08-29 12:56:25 -04:00
Hieu Hoang
f5fc57008f save properties in target phrase object 2013-08-29 13:36:01 +01:00
Lane Schwartz
7675825eb7 Merge branch 'master' of www:/repos/git/Decoders/mosesdecoder 2013-08-28 13:27:12 -04:00
Lane Schwartz
8a726a9de4 Changes to allow SRILM on Mac OS X. 
By default, SRILM defines a function called zopen.

However, on Mac OS X (and possibly other BSDs),
<stdio.h> already defines a zopen function.

To resolve this conflict, SRILM checks to see if HAVE_ZOPEN is defined.
If it is, SRILM will rename its zopen function as my_zopen.

So, before importing any SRILM headers,
it is important to define HAVE_ZOPEN if we are on an Apple OS.
 2013-08-28 12:39:47 -04:00
Hieu Hoang
d79d97c0e6 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-08-28 11:06:44 +01:00
Hieu Hoang
1848946cb3 beautify 2013-08-28 11:06:27 +01:00
Hieu Hoang
89f88fdb6b refactor parsing of args for language models 2013-08-28 11:05:47 +01:00
Lane Schwartz
08225d8e57 Re-enable backward LM code. 
This commit updates that code to take into account Hieu's major refactoring.

Additionally, this commit re-instates changes from
commit 8459a86 (moses/LM/Jamfile, moses/LM/Ken.h, moses/LM/Ken.cpp),
commit d340218 (moses/LM/Ken.h),
commit 95e3a37 (moses/LM/Ken.h, moses/LM/Ken.cpp), and
commit aacc936 (moses/LM/Jamfile, moses/LM/Ken.cpp)

The above changes appear to have been improperly discarded
during the merge in commit 4894df6 and then again in commit 3ed17bb.
 2013-08-27 15:55:07 -04:00
Hieu Hoang
de888f2ab4 store arbitary key-value in translation rules 2013-08-27 09:42:24 +01:00
Kenneth Heafield
1a9a964b68 presize vector 2013-08-26 23:37:37 +01:00
Ulrich Germann
a58efc8ddf Initial check-in. 2013-08-26 16:56:49 +01:00
Nadir Durrani
1444837ff9 Factored-based OSM 2013-08-25 13:23:42 +01:00
Hieu Hoang
f544d3aced mark with LEGACY 2013-08-23 21:34:10 +01:00
Hieu Hoang
83df3629a9 mark GetTargetPhraseCollectionNonCache with LEGACY 2013-08-23 21:08:53 +01:00
Hieu Hoang
de9decc30d mark GetTargetPhraseCollection with LEGACY 2013-08-23 21:02:03 +01:00
Hieu Hoang
5625f37eb1 delete ChartBasedFeatureContext 2013-08-23 15:00:47 +01:00
Hieu Hoang
3ad3cd9004 delete ChartBasedFeatureContext 2013-08-23 14:39:42 +01:00
Hieu Hoang
1d653c87aa delete PhraseBasedFeatureContext 2013-08-23 14:25:25 +01:00
Hieu Hoang
4dfb625536 beautify 2013-08-23 13:53:30 +01:00
Hieu Hoang
0613d98beb delete PhraseBasedFeatureContext 2013-08-23 13:38:31 +01:00
Kenneth Heafield
9186ae2d82 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-08-22 10:21:37 +01:00
Kenneth Heafield
c4dc57f8a3 swap 2013-08-22 10:16:24 +01:00
Hieu Hoang
5accd5f2e5 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2013-08-22 09:24:58 +01:00
Hieu Hoang
21c91dede2 delete GetWordPenaltyProducer(). Need non-const version only for mira 2013-08-22 09:24:46 +01:00
Hieu Hoang
fcf426a098 patch segfault for target phrase caching. Only occurs when multithreading. 2013-08-21 15:58:17 +01:00
Hieu Hoang
9e60195995 patch segfault for target phrase caching. Only occurs when multithreading. 2013-08-21 10:15:29 +01:00
Hieu Hoang
74aea53fd5 caching error for compact pt 2013-08-20 16:06:48 +01:00
Hieu Hoang
51ac2d6567 Merge branch 'hieu_opt_input' of ../hh 2013-08-16 17:05:04 +01:00
Hieu Hoang
4d2fea9b38 prevent segfault in OSM model when used in factored model with decomposed phrase-tables 2013-08-16 16:37:40 +01:00
Hieu Hoang
ae54494417 caching for Marcin's phrase table 2013-08-16 15:33:58 +01:00
Hieu Hoang
41d9386dcc add method to reduce cache size 2013-08-16 15:05:36 +01:00
Hieu Hoang
3bfb0d7a5d caching target phrase for OnDisk pt 2013-08-16 13:47:39 +01:00
Hieu Hoang
077142a2a1 caching target phrase for OnDisk pt 2013-08-16 13:26:21 +01:00
Hieu Hoang
31360e70ef caching target phrase for OnDisk pt 2013-08-16 12:34:31 +01:00
Hieu Hoang
55920d5bca Merge ../mosesdecoder into hieu_opt_input 2013-08-16 12:11:13 +01:00
Hieu Hoang
8363f4afa9 Merge branch 'hieu_opt_input' of github.com:hieuhoang/mosesdecoder into hieu_opt_input 2013-08-16 12:10:36 +01:00
Hieu Hoang
a8c0c0ecde Merge branch 'master' of github.com:hieuhoang/mosesdecoder 2013-08-16 12:06:54 +01:00
Hieu Hoang
21657b6243 delete duplicate parameter 2013-08-16 12:02:48 +01:00
Hieu Hoang
55bd87d38c Merge pull request #43 from jgwinnup/master 
Added option to output unknown words with prefix 'UNK'
 2013-08-15 14:20:48 -07:00
Hieu Hoang
9f92642e59 beautify 2013-08-15 21:14:04 +01:00
Hieu Hoang
49e36cebc2 minor renaming 2013-08-15 20:57:04 +01:00
Hieu Hoang
aa8b8cdc07 move caching code to PhraseDictionary. Available to all phrase tables 2013-08-15 20:50:22 +01:00
Hieu Hoang
841ce108a5 add phrase table caching 2013-08-15 20:20:44 +01:00
Jeremy Gwinnup
091bed03af Added option to output unknown words with prefix 'UNK' 2013-08-15 11:30:08 -04:00
phikoehn
c002a81c96 fixed logic errors 2013-08-15 13:36:56 +01:00
phikoehn
b368085609 xml constraint 2013-08-15 11:46:45 +01:00
Hieu Hoang
abfe02f154 refactor input score handling to use Evaluate(inputtype, inputpath). Not a placeholder factor anymore 2013-08-14 16:53:06 +01:00
Hieu Hoang
2b030339e0 Merge /home/hieu/workspace/github/mosesdecoder into hieu_opt_input 2013-08-14 12:38:26 +01:00
Hieu Hoang
8419a3af35 beautify 2013-08-14 12:37:22 +01:00
Hieu Hoang
bfdedf682b convert weight-p to PhrasePenalty. /Nicola Bertoldi 2013-08-14 12:36:40 +01:00
Hieu Hoang
896b751066 Merge /Users/hieuhoang/unison/workspace/github/mosesdecoder into hieu_opt_input 2013-08-13 21:33:31 +01:00
Hieu Hoang
073b60db35 implement Evaluate(inputpath, input) for chart translation options. Plumbing for lattice input 2013-08-13 20:47:03 +01:00
Hieu Hoang
efc765ddff implement Evaluate(inputpath, input) for chart translation options. Plumbing for lattice input 2013-08-13 20:36:32 +01:00
Hieu Hoang
c50cc18f36 use input path for unknown word processing in phrase-based 2013-08-13 20:01:13 +01:00
Hieu Hoang
563cdc527a use input path for unknown word processing in phrase-based 2013-08-13 19:44:52 +01:00
Hieu Hoang
dcf170a047 refactor bleu score feature function. Still segfaults when running mira 2013-08-13 16:41:35 +01:00
Hieu Hoang
3775e37ee5 refactor bleu score feature function. Still segfaults when running mira 2013-08-13 16:39:37 +01:00
Hieu Hoang
aa7eabf850 add parameter argument -feature-add for mira BLEU score feature 2013-08-13 15:34:03 +01:00
Hieu Hoang
0d60b7f2dd implement Evaluate(inputpath, input) for chart translation options. Plumbing for lattice input 2013-08-13 12:16:55 +01:00
Hieu Hoang
f676100328 implement Evaluate(inputpath, input) for chart translation options. Plumbing for lattice input 2013-08-13 12:12:58 +01:00
Hieu Hoang
af74ee1968 delete duplicate m_inputPathMatrix in TranslationOptionList 2013-08-13 11:44:58 +01:00
Hieu Hoang
4676bca90f minor renaming 2013-08-13 10:33:04 +01:00
Hieu Hoang
154ca336d6 separate out ChartTranslationOption into its own files 2013-08-12 19:01:06 +01:00
Hieu Hoang
9c2c862719 don't hold shared_ptr in RuleCubeItem. Hold only reference to it. Save some speed 2013-08-12 17:35:22 +01:00
Hieu Hoang
c07d8cb43f beautify 2013-08-12 10:35:03 +01:00
Hieu Hoang
2a240b59b2 delete debug messages 2013-08-12 10:34:38 +01:00
Hieu Hoang
46fce4029c create ChartTranslationOption class that store pointer to target phrase. Don't store target phrase itself. QUicker, but still not as quick as storing pointer to target phrase in hypo 2013-08-12 10:34:45 +01:00
Hieu Hoang
8756ad8ffd beautify 2013-08-09 18:17:18 +01:00
Hieu Hoang
a867063a3b convert to using shared pointers in ChartTranslationOptions 2013-08-09 18:16:31 +01:00
Hieu Hoang
4db9aa2973 convert to using shared pointers in ChartTranslationOptions 2013-08-09 15:20:45 +01:00
Hieu Hoang
f4d534ffd7 add InputPath objects of chart decoding. Start on lattice input for chart decoding 2013-08-09 10:27:46 +01:00
Hieu Hoang
59e7a179ca add InputPath objects of chart decoding. Start on lattice input for chart decoding 2013-08-08 21:29:22 +01:00
Hieu Hoang
15132a04aa comment 2013-08-08 17:58:38 +01:00
Hieu Hoang
9341f28d75 Evaluate(InputType) --> Evaluate(InputType, Phrase source) 2013-08-08 17:35:42 +01:00
Hieu Hoang
b044f31012 consistently set source phrase in translation options as soon as the trans opt is created 2013-08-08 17:10:56 +01:00
Hieu Hoang
4b28daeed2 consistently set source phrase in translation options as soon as the trans opt is created 2013-08-08 15:58:01 +01:00
Hieu Hoang
a60bbbf892 consistently set source phrase in translation options as soon as the trans opt is created 2013-08-08 15:03:01 +01:00
Hieu Hoang
88f888f534 const TargetPhrase in TargetPhraseCollection 2013-08-08 14:38:41 +01:00
Hieu Hoang
9062194f46 comments 2013-08-08 12:26:25 +01:00
Hieu Hoang
f70084178c Merge github.com:moses-smt/mosesdecoder into hieu_opt_input 2013-08-07 15:53:42 +01:00
phikoehn
e50fc722e9 bug fix alternative weight setting 2013-08-07 15:35:40 +01:00
Hieu Hoang
6f7d239f13 Merge github.com:moses-smt/mosesdecoder into hieu_opt_input 2013-08-07 15:19:22 +01:00
Hieu Hoang
585ec0f7fe DONE: delete source phrase from TargetPhrase class 2013-08-07 15:18:45 +01:00
Hieu Hoang
0ab6a441b2 get ready to delete source phrase from TargetPhrase class 2013-08-07 15:02:36 +01:00
Hieu Hoang
b3a860fd4f get ready to delete source phrase from TargetPhrase class 2013-08-07 14:35:40 +01:00
Hieu Hoang
441acf15e6 beautify 2013-08-07 14:18:12 +01:00
Hieu Hoang
5eef91c0ea get ready to delete source phrase from TargetPhrase class 2013-08-07 14:10:42 +01:00
Hieu Hoang
71073868a8 PhraseDictionaryTreeAdaptor::GetTargetPhraseCollectionLegacy return tpColl and vector of source phrases 2013-08-07 12:55:10 +01:00
Hieu Hoang
5deefbddaf renaming a function 2013-08-07 12:34:31 +01:00
Hieu Hoang
24c4c8f210 mark a function as legacy 2013-08-07 12:11:39 +01:00
Hieu Hoang
89f39bf89d mark a function as legacy 2013-08-07 11:48:18 +01:00
Hieu Hoang
a1f5167f85 get ready to delete source phrase from TargetPhrase class 2013-08-07 11:37:40 +01:00
Hieu Hoang
66d4c2b0be source phrase is a pointer in translation options. 2013-08-07 10:28:28 +01:00
phikoehn
67c3063574 Merge branch 'master' of ssh://github.com/moses-smt/mosesdecoder 2013-08-07 05:32:59 +01:00
phikoehn
ab4e3c63a6 enriched trace 2013-08-07 05:31:45 +01:00
Hieu Hoang
7a808a2edb push InpuutPath down into phrase dictionary for legacy methods 2013-08-06 17:09:53 +01:00