moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-12-28 14:32:38 +03:00
Code Issues Packages Projects Releases Wiki Activity
9,600 Commits 136 Branches 26 Tags 147 MiB
18e8f12d5e
Commit Graph

2114 Commits

Author SHA1 Message Date
Hieu Hoang
feff7affaf add tests for decoding with compressed pt 2012-11-20 16:14:20 +00:00
Hieu Hoang
dbf0123c5c don't load compact lex model if it doesn't exist 2012-11-20 13:22:03 +00:00
Hieu Hoang
0bf06666b1 add error message for non-existant input file 2012-11-20 12:11:01 +00:00
Marcin Junczys-Dowmunt
f09fb75be1 Compact reordering table, do not force full recompilation 2012-11-18 10:21:35 +01:00
Marcin Junczys-Dowmunt
e54360840f Do not search for compact reordering table if CMPH is not compile into moses 2012-11-17 22:03:49 +01:00
marcinj
79526d0a6b more consistent file name resolution 2012-11-17 10:07:33 +01:00
marcinj
5a010cb1d5 Revert "Binary compatible compact phrase table" 
This reverts commit ad8fbc807e.
 2012-11-17 00:46:22 +01:00
marcinj
ad8fbc807e Binary compatible compact phrase table 2012-11-17 00:40:07 +01:00
Hieu Hoang
333da4366f Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-11-16 17:05:01 +00:00
Marcin Junczys-Dowmunt
26d39408f0 Fixed destructor PhraseDictionaryCompact 2012-11-16 17:52:20 +01:00
Marcin Junczys-Dowmunt
58f358ea2e fixed destructor 2012-11-16 17:49:31 +01:00
Hieu Hoang
90b251a50f Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-11-16 16:44:56 +00:00
Hieu Hoang
057793fc60 output alignments into file down. Alignments for n-best todo 2012-11-16 16:44:29 +00:00
Marcin Junczys-Dowmunt
e098a1fa9e Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-11-16 17:44:13 +01:00
marcinj
0edd6b4a61 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-11-16 17:37:38 +01:00
Marcin Junczys-Dowmunt
a9017269df Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-11-16 17:33:54 +01:00
marcinj
f858f919b3 Fixed missing alignment points 2012-11-16 17:33:20 +01:00
Kenneth Heafield
4f3afbf41f Merge branch 'master' of github.com:moses-smt/mosesdecoder 
Conflicts:
	moses-chart-cmd/IOWrapper.cpp
	moses-chart-cmd/Main.cpp
 2012-11-16 16:07:14 +00:00
Kenneth Heafield
2df57b6b09 Expose k-best lists from incremental search 2012-11-16 15:53:43 +00:00
Marcin Junczys-Dowmunt
57e74959bf Use copy of phrase table weights instead of pointer 2012-11-16 16:25:54 +01:00
Barry Haddow
7d74673fbe Short circuit feature pre-calc when possible 2012-11-16 10:14:22 +00:00
Kenneth Heafield
6a3fdcd8c0 Rename Incremental/Manager.cpp to just Incremental.cpp 2012-11-15 22:10:30 +00:00
Kenneth Heafield
9340f9f465 Update incremental search 2012-11-15 18:04:07 +00:00
Hieu Hoang
7f0b58f0a3 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-11-15 16:43:18 +00:00
Hieu Hoang
ac8d6b1d7b added alignment output to n-best list 2012-11-15 16:43:01 +00:00
Hieu Hoang
f6d3f8592b added plumbing to output alignments from chart decoder. 
Currently outputs to alignment file, output to n-best list still to do.
Haven't adjusted for source and target offset, that's gonna be difficult
 2012-11-15 16:42:19 +00:00
Barry Haddow
a90e1861c0 Alignments on by default for phrase-based 2012-11-15 12:35:43 +00:00
Hieu Hoang
c916206e84 if you don't need word alignments, but the binary phrase table has it anyway. Don't throw a fit 2012-11-14 19:48:08 +00:00
Hieu Hoang
78b3c8763b delete arg -use-alignment-info 2012-11-14 19:01:25 +00:00
Hieu Hoang
623289465c change argument -include-alignment-in-n-best to -include-segmentation-in-n-best. Avoids confusion with word alignment 2012-11-14 16:57:33 +00:00
Hieu Hoang
1a596181d1 change argument -include-alignment-in-n-best to -include-segmentation-in-n-best. This is what it actually does 2012-11-14 16:47:16 +00:00
Hieu Hoang
4e9d2f3ae1 delete print-alignment-info arg. Not used anymore 2012-11-14 16:43:06 +00:00
Hieu Hoang
065597dc9f only include moses root when compiling 2012-11-14 14:18:53 +00:00
Hieu Hoang
e2841edfed Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-11-14 13:43:17 +00:00
Hieu Hoang
f96b33de83 only include moses root when compiling 2012-11-14 13:43:04 +00:00
Barry Haddow
462c04b9f7 remove unused parameter 2012-11-14 10:37:24 +00:00
Hieu Hoang
bc64acaaa4 remove hardcoded label for top of sentence 2012-11-13 17:43:52 +00:00
Kenneth Heafield
f8e1856022 Redo dynsa cleanup 2012-11-13 11:30:58 +00:00
Kenneth Heafield
d19463b77e Cleanup a file left by a merge 2012-11-13 11:28:31 +00:00
Hieu Hoang
44f24bd673 resolved merge conflicts 2012-11-12 20:35:32 +00:00
Hieu Hoang
5e3ef23cef move moses/src/* to moses/ 2012-11-12 19:56:18 +00:00
Kenneth Heafield
d4f433c7f4 Put core Moses into one library libmoses. 2012-11-12 18:07:53 +00:00
Hieu Hoang
8c785cff2b add header file 2012-11-12 17:21:02 +00:00
Hieu Hoang
ac4a75a0af c++ version of fuzzy match works 2012-11-12 16:37:27 +00:00
Hieu Hoang
ea24000a84 c++ version of fuzzy match works 2012-11-12 13:58:39 +00:00
Kenneth Heafield
4f8f864650 Instant upgrade to a write lock (exclusive first class seat) 2012-11-12 12:49:34 +00:00
Kenneth Heafield
e3e3ad4c0d cmph header makes it further than CompactPT 2012-11-09 08:44:09 +00:00
Kenneth Heafield
9b8254a90e Collapse a bunch of Moses libraries into one. 2012-11-08 14:17:15 +00:00
Kenneth Heafield
e9eb7dd021 More shared build fixes 2012-11-07 23:28:42 +01:00
Kenneth Heafield
9a1ac30889 Increase paranoia about --kenlm-max-order and --max-factors 2012-10-31 14:48:29 +00:00
Hieu Hoang
deb694fb7c minor compile error on macosx. size_t != uint64_t 2012-10-30 14:56:20 -07:00
Kenneth Heafield
b0a04ae50e Cut PhraseTableMemory loading time by 35% 
Before:
user    122.046
sys     3.029
VmPeak: 13091604 kB
VmRSS:  13052228 kB

After double conversion:
user    117.389
sys     3.330
VmPeak: 13091640 kB
VmRSS:  13052204 kB

Plus eliminate boost lexical cast:
user    82.632
sys     3.177
VmPeak: 13088560 kB
VmRSS:  13049056 kB

Plus inline scoreVector:
user    79.396
sys     2.892
VmPeak: 13088560 kB
VmRSS:  13049064 kB
 2012-10-28 17:47:22 +00:00
Hieu Hoang
e35bb2c81f delete some debug info 2012-10-25 14:14:15 +01:00
Hieu Hoang
7a71270be2 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-10-24 10:38:05 +01:00
Hieu Hoang
fb8eb0d111 fix marcin's querying of compact rule table 2012-10-24 10:34:28 +01:00
Kenneth Heafield
73ef77163c Remove m_ruleCount, construct source phrase in-place 2012-10-23 19:16:50 +01:00
Hieu Hoang
ff0946dd3e eclipse files 2012-10-23 17:07:28 +01:00
Hieu Hoang
9a9dfe59de default to multiple matched 2012-10-23 17:07:12 +01:00
Kenneth Heafield
eb28ea26a8 Add a phrase default constructor, bypassing the array size increment 2012-10-22 13:17:46 -04:00
Kenneth Heafield
2e8c316aaa Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-10-22 17:42:37 +01:00
Kenneth Heafield
c34ecf12de TargetPhrase doesn't have a FactorDirection constructor. 
It was implicitly doing TargetPhrase(Phrase(size_t)) wastefully
 2012-10-22 17:40:23 +01:00
Hieu Hoang
87b6520d82 add moses server back in. Changed ScoreComponentCollection to expose dense feature array 2012-10-22 14:41:23 +01:00
Hieu Hoang
cb0b94fad5 delete all mention of terminal in word alignment class 2012-10-19 16:00:42 +01:00
Hieu Hoang
f4c965e752 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-10-19 15:10:20 +01:00
Hieu Hoang
397e704b49 terminal and non-terminal word alignments stored in 2 separate objects 2012-10-19 15:10:10 +01:00
Kenneth Heafield
791a7b68fc Make max-factors a compile option and add recompilation safety 2012-10-19 10:16:45 +01:00
Kenneth Heafield
39664ac8f4 Support arbitrary arity 
Also, some windows fixes in util.
 2012-10-18 18:54:38 +01:00
Hieu Hoang
ee47e53065 new weight vector in fuzzy match rule table 2012-10-18 16:19:47 +01:00
Kenneth Heafield
3ced55e33d Put note into PartialEdge, delete edge_queue 
Before with pop 25:
real    9m49.480s
user    9m38.389s
sys     0m10.354s

After with pop 25:
real    8m46.553s
user    8m35.138s
sys     0m10.654s
 2012-10-17 16:49:41 +01:00
Kenneth Heafield
4cf1ffb7d8 Simplify SetRuleCount 2012-10-17 11:48:30 +01:00
Barry Haddow
ed9d8c4992 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-10-17 11:22:40 +01:00
Barry Haddow
a642705ed3 rule count for glue rule 2012-10-17 11:22:09 +01:00
Barry Haddow
befe18351f OnDisk phrase table is not thread-safe! 
And declaring it to be so does not make it so.

Also, ensure that the phrase dictionary is loaded before
it is used.
 2012-10-17 11:18:38 +01:00
Kenneth Heafield
9f232895f3 More loading optimization 
Cleanup Phrase substr, optimistic single-threaded FactorCollection lookup

Before:
user    242.228
sys     4.890
VmPeak: 17294456 kB
VmRSS:  17235148 kB

After:
user    215.322
sys     5.029
VmPeak: 17294544 kB
VmRSS:  17235304 kB
 2012-10-17 11:07:42 +01:00
Kenneth Heafield
8ebaf44ae8 tmpnam is evil 2012-10-17 10:06:38 +01:00
Kenneth Heafield
63dda2a79d Merge remote branch 'upstream/master' 2012-10-16 18:21:07 -04:00
Kenneth Heafield
0340026ca2 Got tired of loading time. Again. 
After miramerge went in:
real    6m8.948s
user    5m57.609s
sys     0m10.823s

Now:
real    4m27.469s
user    4m16.311s
sys     0m10.605s
 2012-10-16 18:18:57 -04:00
Hieu Hoang
13249c54e5 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2012-10-16 19:01:10 +01:00
Hieu Hoang
bd9f4c282e explicit operator= . Required for gcc 4.6.3 but not 4.5.0 2012-10-16 19:00:41 +01:00
Kenneth Heafield
43fcd44a1d Merge remote branch 'upstream/master' 2012-10-16 12:59:18 -04:00
Kenneth Heafield
b9d228b281 Fix compiler errors for more strict compilers 2012-10-16 12:58:38 -04:00
Hieu Hoang
275dca4be1 explicit operator= . Required for gcc 4.6.3 but not 4.5.0 2012-10-16 17:50:00 +01:00
Kenneth Heafield
f9f5b4eb0a Merge remote branch 'upstream/master' 2012-10-16 17:23:17 +01:00
Hieu Hoang
f0714bb1bc minor compile error 2012-10-16 16:57:06 +01:00
Hieu Hoang
a6f63af172 same version number check when reading and writing 2012-10-16 16:26:31 +01:00
Barry Haddow
42d74f2306 Fix regression test 2012-10-16 15:23:26 +01:00
Kenneth Heafield
0378a58a37 Start arbitrary arity. Currently incorrect. 2012-10-16 12:57:18 +01:00
Barry Haddow
365e680115 Merge remote-tracking branch 'origin/master' into miramerge 
NB Untested

Conflicts:
	Jamroot
	moses-chart-cmd/src/Main.cpp
	moses/src/ChartManager.cpp
	moses/src/RuleTable/LoaderStandard.cpp
	moses/src/RuleTable/PhraseDictionaryALSuffixArray.cpp
	moses/src/Word.cpp
 2012-10-15 21:35:56 +01:00
Kenneth Heafield
0eb98df0fe Submodules are evil. Undo it. 2012-10-15 13:58:33 +01:00
Kenneth Heafield
ed00315c8c Remove Owner class 2012-10-15 06:43:43 -04:00
Kenneth Heafield
0265725c03 Merge 2012-10-14 16:02:04 -04:00
Kenneth Heafield
44eee4dcd2 FilePieceify LoaderStandard. 2012-10-14 20:52:12 +01:00
Kenneth Heafield
a0ce62e795 Got tired of waiting for loading. 2012-10-14 17:35:58 +01:00
Kenneth Heafield
0995fa52a8 Use non-terminal alignments 2012-10-14 16:05:29 +01:00
Barry Haddow
61ae24aa5d Merge remote-tracking branch 'origin/master' into miramerge 
Conflicts:
	moses/src/PhraseDictionary.cpp
	moses/src/TargetPhrase.cpp
	moses/src/TargetPhrase.h
 2012-10-14 14:18:03 +01:00
Kenneth Heafield
031995d897 Insert blanks properly 2012-10-13 20:49:49 +01:00
Kenneth Heafield
64f91af29d Refactor to streamline edge creation 2012-10-13 20:32:55 +01:00
Kenneth Heafield
8ed542cf29 Merge remote branch 'upstream/master' 
Conflicts:
	moses/src/TargetPhrase.h
 2012-10-13 19:39:59 +01:00
Kenneth Heafield
0d9095983d Remove tmodel from TargetPhrase 2012-10-13 19:30:39 +01:00
Kenneth Heafield
42f009d86d Hacky intermediate state 2012-10-13 18:42:42 +01:00