moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-09-20 23:58:15 +03:00
Code Issues Packages Projects Releases Wiki Activity
6,705 Commits 136 Branches 26 Tags 147 MiB
f99c35395c
Commit Graph

1371 Commits

Author SHA1 Message Date
Eva Hasler
f99c35395c Merge remote branch 'github/miramerge' into miramerge 2011-11-09 15:41:33 +00:00
Barry Haddow
818b2594c1 Fix compile error with mpi 2011-11-08 09:43:04 +00:00
Barry Haddow
daeed698c1 Faster with copy than += 2011-11-07 10:54:53 +00:00
Eva Hasler
905dcaeeb0 Merge branch 'miramerge' of ssh://mosesdecoder.git.sourceforge.net/gitroot/mosesdecoder/mosesdecoder into miramerge 2011-11-04 16:43:39 +00:00
Eva Hasler
aa32150525 adding class TargetNgramFeature, produces ngrams incl. or excl. lower order ngrams 2011-11-04 16:40:12 +00:00
Barry Haddow
42a3f28b42 Speed up decoding by cutting reducing score copies. 
Hypothesis gets the weighted score from previous,
and lazily computes full breakdown. Changes lex
reorder scores very slightly (third decimal place),
hence test change.
 2011-11-03 22:33:05 +00:00
Barry Haddow
90820ad0c5 Merge branch 'master' into miramerge. 
Also fix mert-moses.pl to use correct flag for specifying
weights of non-core features.

Conflicts:
	.gitignore
	configure.in
	ltmain.sh
	moses/src/LM/Factory.cpp
	moses/src/LMList.cpp
	moses/src/LMList.h
	moses/src/LanguageModel.cpp
	moses/src/LanguageModel.h
	moses/src/LanguageModelKen.h
	moses/src/Makefile.am
	moses/src/Manager.cpp
	moses/src/PhraseDictionaryMemory.cpp
	moses/src/PhraseDictionaryTree.cpp
	moses/src/StaticData.cpp
	moses/src/TargetPhrase.h
 2011-10-28 15:54:23 +01:00
Hieu Hoang
42924144fd Merge branch 'master' of github.com:moses-smt/mosesdecoder 2011-10-28 19:13:14 +07:00
Hieu Hoang
62c901eb52 TMX extraction by Tom Hoar and Hilario Leal Fontes 2011-10-28 19:12:20 +07:00
Germán
899293243a Modified code in Manager.cpp so that option -osgx outputs a superset of -osg. 2011-10-27 13:29:46 +02:00
Eva Hasler
7bbcc67344 scale by average of source and reference length (--scale-by-avg-length) 2011-10-26 11:36:00 +01:00
Eva Hasler
7d0a5fa11f scale Bleu by complete source length or reference length 2011-10-26 11:16:45 +01:00
Eva Hasler
7a5637f803 fix new_state->m_source_length (?) 2011-10-25 16:30:02 +01:00
Eva Hasler
4ab2db98fe add parameter --scale-by-x: scale Bleu precision (independent of source/target scaling) 2011-10-24 18:39:23 +01:00
Hieu Hoang
d98780c062 Just get ready but didn't fix bug yet 2011-10-24 19:13:56 +07:00
Hieu Hoang
ae3ecbc105 fix bug for tree-to-string. Didn't check source LHS 2011-10-24 19:13:17 +07:00
Hieu Hoang
82e2e094ff fix bug for tree-to-string. Didn't check sourceLHS 2011-10-24 18:54:42 +07:00
Eva Hasler
e7c8120bf6 new parameter: scale by target length instead of input length 2011-10-24 10:43:53 +01:00
Hieu Hoang
cb87f251b2 xcode 2011-10-24 14:44:36 +07:00
Hieu Hoang
f8b6387642 xcode 2011-10-24 01:54:25 +07:00
Hieu Hoang
a93f4691f6 win32 2011-10-23 09:37:47 +07:00
Eva Hasler
7e5b3fa061 calculate length ratios, add +0.1 smoothing and papineni smoothing 2011-10-22 22:23:58 +01:00
Eva Hasler
eebfd61e48 re-apply changes unrelated to setting reference translations 2011-10-20 14:32:05 +01:00
Eva Hasler
b263f14f63 this broke something, reverting.. 
Revert "fix LoadReferences() in StaticData and remove reference loading in mira's Main and Decoder"

This reverts commit 2ee9f47a5f.
 2011-10-20 13:43:30 +01:00
Hieu Hoang
1120754a9b xcode compiles 2011-10-18 19:45:52 +07:00
Hieu Hoang
f2814c65b9 xcode. kenlm fails for some reason 2011-10-18 19:28:50 +07:00
Kenneth Heafield
8ec57a4c32 Now PhraseDictionaryMemory loads in 36% of the original time. 2011-10-17 12:38:53 +01:00
heafield
f0be9d9cf0 More Boost is allowed partying 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4381 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-17 09:40:46 +00:00
heafield
68a4626a49 Remove reference counts now that we can use boost 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4380 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-17 09:30:30 +00:00
heafield
2bb2d6dc4a Reduce text phrase table loading time by 49.5%. Add a progress bar too. StringPiece is good for you. 
This change introduces a dependency on Boost, which is now permitted in Moses.  



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4365 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-14 16:40:30 +00:00
hieuhoang1972
897fe0f88b visual studio 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4356 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-14 10:50:08 +00:00
Barry Haddow
235f737c76 Fix regression tests. 
Binary format now backwards compatible.
Fix LM oov feature.
 2011-10-13 17:50:16 +01:00
heafield
7ead82ba41 Remove extraneous header 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4353 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-13 16:22:04 +00:00
heafield
6b153c67f8 (16:51:52) Heafield: Does anybody use LanguageModelSkip? 
(16:52:12) Hieu Hoang: not since jhu 2006
(16:52:17) Heafield: svn rm?  
(16:52:34) Hieu Hoang: aye. & see if anyone complains
(16:52:49) Hieu Hoang: & internal if u want to



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4352 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-13 16:01:00 +00:00
heafield
6bded791e6 Remove some virtual tags 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4351 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-13 15:34:37 +00:00
heafield
07e611ebcb Organize language models into an LM directory. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4350 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-13 14:27:01 +00:00
heafield
a95e791056 Back to using StringPiece 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4349 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-13 13:32:14 +00:00
heafield
f084248405 Cut the middle men out of the language model interface. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4348 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-13 12:33:05 +00:00
heafield
7d9bc523a6 Remove unused code 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4347 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-13 09:44:51 +00:00
heafield
541f776198 Remove unused calls 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4346 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-12 20:04:02 +00:00
heafield
e5d15a537e KenLM-specific Evaluate function 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4345 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-12 19:49:27 +00:00
Barry Haddow
c83166087e Merge branch 'master' into miramerge 
Conflicts:
	moses/src/LanguageModel.cpp
	moses/src/TargetPhrase.h
	moses/src/TrellisPath.h
	moses/src/Util.h
	scripts/training/train-model.perl
 2011-10-12 17:14:23 +01:00
heafield
cd19f14826 Faster CalcScore implementation for KenLM 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4339 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-12 13:04:12 +00:00
heafield
81acd0ffa2 Dear Hieu, a StringPiece is not necessairly null-terminated. When loading ARPA files directly, it was copying the ARPA file as 
part of the vocabulary word and breaking everything.  



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4338 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-12 11:45:46 +00:00
heafield
c3f2ef7b25 Fix bhaddow's oovCount. Should be all words, not just the first in the phrase. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4337 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-12 10:22:45 +00:00
heafield
15adb17e35 Move EnumerateVocab to namespace lm 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4335 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-12 10:18:23 +00:00
hieuhoang1972
a65efa5a60 relax overly harsh assert 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4334 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-12 10:12:49 +00:00
Eva Hasler
51b54e0d9d Merge branch 'miramerge' of thor.inf.ed.ac.uk:/fs/saxnot3/ehasler/mosesdecoder_git_mira into miramerge 2011-10-11 18:10:00 +01:00
Eva Hasler
100007b85b fix LoadReferences() in StaticData and remove reference loading in mira's Main and Decoder, fix decoding-graph-backoff parameter 2011-10-11 18:09:39 +01:00
Eva Hasler
2ee9f47a5f fix LoadReferences() in StaticData and remove reference loading in mira's Main and Decoder 2011-10-11 17:49:08 +01:00