moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2025-01-02 17:09:36 +03:00
Code Issues Packages Projects Releases Wiki Activity
16,684 Commits 136 Branches 26 Tags 147 MiB
259cb96538
Commit Graph

2089 Commits

Author SHA1 Message Date
Hieu Hoang
0008440e25 use pool. Leak fixed 2016-06-21 23:59:01 +01:00
Hieu Hoang
350ba32eb7 use pool 2016-06-21 23:53:50 +01:00
Hieu Hoang
996df0c984 get ready to use pool 2016-06-21 23:36:58 +01:00
Hieu Hoang
8614486cdf merge 2016-06-21 21:28:44 +01:00
Hieu Hoang
fa2e28dd4d comment out debugging info 2016-06-21 21:26:43 +01:00
Hieu Hoang
3122574b40 don't use Debug() to output to nbest. Take out total score from nbest score breakdown 2016-06-21 21:19:25 +01:00
Hieu Hoang
5118578347 ToString() -> OutputTargetPhrase() 2016-06-21 21:07:38 +01:00
Hieu Hoang
c4daa158c2 move ReorderingConstraint to PhraseBased/ 2016-06-21 18:17:15 +01:00
Hieu Hoang
320f01c722 merge 2016-06-21 17:41:53 +01:00
Hieu Hoang
78dee8201e merge 2016-06-21 17:39:00 +01:00
Hieu Hoang
cf59f24654 xml constraint not supported 2016-06-21 16:57:16 +01:00
Hieu Hoang
6570fa969e set m_xmlCoverageMap 2016-06-21 16:52:10 +01:00
Hieu Hoang
23677d2b65 compile errors 2016-06-21 15:24:58 +01:00
Hieu Hoang
501f9ccd8d use xml options 2016-06-21 15:08:18 +01:00
Hieu Hoang
44960ca13d Merge ../mosesdecoder.perf.public into perf_moses2 2016-06-21 14:10:20 +01:00
Hieu Hoang
eacd2cb5ae use xml options 2016-06-21 14:10:13 +01:00
Hieu Hoang
062081c2a9 use xml options 2016-06-21 12:46:46 +01:00
Hieu Hoang
b1a0ca960d parse xml forced translation 2016-06-21 12:35:16 +01:00
Hieu Hoang
ac8fbbde40 add SatisfyBackoff(). Supports XML too 2016-06-21 11:20:38 +01:00
Hieu Hoang
f40c6bd030 separate out xml processing into separate method 2016-06-20 19:11:53 +01:00
Hieu Hoang
fc02b00360 expose unkWP FF. To be used to put XML forced translation 2016-06-20 19:07:50 +01:00
Hieu Hoang
fb2bbf99fc debug 2016-06-20 16:33:04 +01:00
Hieu Hoang
19b8bc50b7 test signing 
Signed-off-by: Hieu Hoang <hieuhoang@gmail.com>
 2016-06-18 01:15:40 +01:00
Hieu Hoang
2f9d8d211f debug 2016-06-17 23:06:02 +01:00
Hieu Hoang
ad2153f264 debug 2016-06-17 22:54:32 +01:00
Hieu Hoang
70ce10a85e debug 2016-06-17 21:55:47 +01:00
Hieu Hoang
12a56efe19 GetKey can be pass pt oov 2016-06-17 21:12:17 +01:00
Hieu Hoang
eb076ed392 lookup scfg rules in probing pt 2016-06-17 20:13:41 +01:00
Hieu Hoang
9de1f72a8e get ready to hash source 2016-06-17 16:35:17 +01:00
Hieu Hoang
45790f3c38 get ready to hash source 2016-06-17 16:09:21 +01:00
Hieu Hoang
42b3f337fd get ready to hash source 2016-06-17 15:11:59 +01:00
Hieu Hoang
b64e3afd4d minor rename 2016-06-17 14:31:30 +01:00
Hieu Hoang
96158051e7 don't use last source word when storing. It's the LHS 2016-06-17 14:21:19 +01:00
Hieu Hoang
774b4dd78a cleanup 2016-06-17 13:50:43 +01:00
Hieu Hoang
7d00c6a8fe eclipse 2016-06-17 12:56:57 +01:00
Hieu Hoang
aa07fd26cf eclipse 2016-06-17 12:48:28 +01:00
Hieu Hoang
e7ec90da58 start scfg decoding with probing pt 2016-06-16 18:17:48 +01:00
Hieu Hoang
62c23a9859 start scfg decoding with probing pt 2016-06-16 17:14:18 +01:00
Hieu Hoang
cad0235159 reformat NT 2016-06-16 15:24:34 +01:00
Hieu Hoang
ca264d8fcd retrieve non-terms 2016-06-16 13:53:20 +01:00
Hieu Hoang
647a9a3cff start reformat rules for storing 2016-06-16 12:41:50 +01:00
Hieu Hoang
127c057e62 use StoreVocab for source vocab, take 2. Continue to use simple hashing 2016-06-16 11:26:30 +01:00
Hieu Hoang
101b4bfe0c modularize getKey() 2016-06-16 10:41:47 +01:00
Hieu Hoang
f8f916aeba Revert "use StoreVocab for source vocab" 
This reverts commit 7f39875c74.
 2016-06-16 05:05:17 -04:00
Hieu Hoang
7f39875c74 use StoreVocab for source vocab 2016-06-15 19:41:03 +01:00
Hieu Hoang
193c921281 use template type 2016-06-15 18:25:34 +01:00
Hieu Hoang
30bd411e17 add GetVocabIds() 2016-06-15 18:21:04 +01:00
Hieu Hoang
73ac4e43cb use StoreVocab class in StoreTarget 2016-06-15 17:40:39 +01:00
Hieu Hoang
c068365d85 add StoreVocab class 2016-06-15 17:05:11 +01:00
Hieu Hoang
6ec42385ac start scfg for probing pt 2016-06-15 15:24:30 +01:00
Hieu Hoang
d57aa3c40b better error message 2016-06-15 14:53:48 +01:00
Hieu Hoang
161f80dc92 better error message 2016-06-15 14:48:43 +01:00
Hieu Hoang
661fab4ea5 std::unordered_map -> boost::unordered_map 2016-06-15 14:45:12 +01:00
Hieu Hoang
3ed2bf1a52 update probing pt version. Change config format 2016-06-15 12:56:36 +01:00
Hieu Hoang
fbb8b30681 Everything in SeenPosition is also in QueueItem so make 1 contain the other 2016-06-15 10:49:24 +01:00
Hieu Hoang
faa92e8c5e tpInd -> tpInd + 1 2016-06-15 01:12:26 +01:00
Hieu Hoang
141f7a246c SeenPositionItem -> SeenPosition 2016-06-15 01:05:35 +01:00
Hieu Hoang
045903ff79 support output-hypo-score arg. std::vector -> Vector 2016-06-15 01:01:29 +01:00
Hieu Hoang
081639e536 index SeenPosition by tps and tpInd, not by tp. Same tp could span different non-terms 2016-06-15 00:47:00 +01:00
Hieu Hoang
c8539ea61e debug 2016-06-15 00:34:30 +01:00
Hieu Hoang
cfebaabc73 debug 2016-06-14 22:45:19 +01:00
Hieu Hoang
9942fec457 use pointer for SeenPositionItem 2016-06-14 21:34:07 +01:00
Hieu Hoang
572d7aec03 debug 2016-06-14 17:22:19 +01:00
Hieu Hoang
be23e00e13 debug 2016-06-14 17:14:48 +01:00
Hieu Hoang
1674159c43 debug info 2016-06-13 16:58:08 +01:00
Hieu Hoang
fd1c0d891f debug info 2016-06-13 16:16:51 +01:00
Hieu Hoang
416e66e9fe don't create rules with unk pt if there are already rules 2016-06-13 15:33:26 +01:00
Hieu Hoang
0e8111e422 sort by p(e|f) 2016-06-13 14:34:50 +01:00
Hieu Hoang
8f4764f1bc Change CompareFutureScore() 2016-06-13 13:59:26 +01:00
Hieu Hoang
14bfbdc5e3 change pointer to object 2016-06-13 13:47:15 +01:00
Hieu Hoang
91b01b55a0 cleanup Score class 2016-06-11 02:12:48 +01:00
Hieu Hoang
e23041f7a8 separate Debug() and OutputToStream() 2016-06-11 02:07:43 +01:00
Hieu Hoang
d90a31d8dd consistent Debug() 2016-06-11 01:51:26 +01:00
Hieu Hoang
4453aecb2b consistent Debug() 2016-06-11 01:45:26 +01:00
Hieu Hoang
04fd6630d5 consistent Debug() 2016-06-11 01:31:40 +01:00
Hieu Hoang
f3874d3945 consistent Debug() 2016-06-11 01:03:23 +01:00
Hieu Hoang
7f8c66df7a fix segfault 2016-06-11 00:55:39 +01:00
Hieu Hoang
bc77f63948 consistent debug function 2016-06-11 00:15:15 +01:00
Hieu Hoang
5a69037115 oov penalty 2016-06-09 23:12:58 +01:00
Hieu Hoang
f8b6e55bfc add scores to new hypo 2016-06-09 22:55:59 +01:00
Hieu Hoang
ea1517bd77 print memory usage 2016-06-09 17:17:20 +01:00
Hieu Hoang
482457bf2b align to 4-byte boundary. Seems to work. 2016-06-08 09:06:39 -04:00
Hieu Hoang
f8c0c25400 revert previous & fix bug 2016-06-08 13:26:40 +01:00
Hieu Hoang
0da2cbc214 AllocateWithAlign() aligns each object in array to 8 byte boundary 2016-06-08 13:21:15 +01:00
Hieu Hoang
0958d3f4f4 size_t -> bool. Use AllocateWithAlign() to allocate mem for bool 2016-06-08 13:10:14 +01:00
Hieu Hoang
e85f048ec9 add AllocateWithAlign() to MemPool 2016-06-08 13:07:09 +01:00
Hieu Hoang
dc052be4ca bool -> size_t. Prevent segfault when using mempool 2016-06-08 12:55:53 +01:00
Hieu Hoang
00618430fa std::vector -> Vector 2016-06-07 18:07:49 +01:00
Hieu Hoang
9cf2862a52 vector -> pair 2016-06-07 18:04:12 +01:00
Hieu Hoang
15b8c97fce use mempool to stem leak 2016-06-07 17:57:33 +01:00
Hieu Hoang
849b161bf8 make XMLOption an inner class of Sentence 2016-06-07 17:39:01 +01:00
Hieu Hoang
c39dbcb17b cleanup 2016-06-07 17:33:20 +01:00
Hieu Hoang
81642a82c1 debugging zones. Work the same as moses 2016-06-07 17:23:31 +01:00
Hieu Hoang
462b17de8a use zone and walls for normal and batch algorithm 2016-06-07 16:08:31 +01:00
Hieu Hoang
b63380dae4 use zone and walls 2016-06-07 15:59:18 +01:00
Hieu Hoang
36dedae35f zone 2016-06-07 14:09:50 +01:00
Hieu Hoang
7c038657aa compile error on xdata 2016-06-07 13:46:22 +01:00
Hieu Hoang
764b2b2e05 import ReorderingConstraint class 2016-06-07 13:40:32 +01:00
Hieu Hoang
4fcc10a2c2 don't create xml options if we don't need it 2016-06-06 20:42:26 +01:00
Hieu Hoang
66493f9718 start xml 2016-06-06 20:27:48 +01:00
Hieu Hoang
05a2c0c1ab recycle queue items 2016-06-05 23:37:13 +01:00
Hieu Hoang
9442e26473 recycle queue items 2016-06-05 23:29:27 +01:00
Hieu Hoang
7b328f3cdc recycle hypos 2016-06-05 21:35:38 +01:00
Hieu Hoang
81dd25ec0a seen positions 2016-06-04 01:17:46 +01:00
Hieu Hoang
5a065fa2c6 seen positions 2016-06-03 23:53:40 +01:00
Hieu Hoang
fec1319e33 consistent use of mem pool 2016-06-03 23:04:02 +01:00
Hieu Hoang
77f0551008 seen positions 2016-06-03 22:50:12 +01:00
Hieu Hoang
aab5ccd1c7 delete debug 2016-06-03 21:26:27 +01:00
Hieu Hoang
821153c75d clear out queue before using it 2016-06-03 21:21:42 +01:00
Hieu Hoang
d0acadae46 debug 2016-06-03 19:57:51 +01:00
Hieu Hoang
d0c237e2d6 don't require n-best list 2016-06-03 17:04:30 +01:00
Hieu Hoang
1a2edfbfeb fix leaks 2016-06-03 16:19:22 +01:00
Hieu Hoang
6206ee8d40 get ready to make QueueItem a recycleable object 2016-06-03 16:03:41 +01:00
Hieu Hoang
0702f9fb79 share queue item's HyposColl 2016-06-03 15:53:33 +01:00
Hieu Hoang
dbaac76c1f only create hypos with static Create() 2016-06-03 15:33:07 +01:00
Hieu Hoang
408490f626 call EvaluateWhenApplied(). No segfaults. Hallelujah 2016-06-03 15:29:11 +01:00
Hieu Hoang
2a2fcc9696 move symbol bind from hypo to queue item 2016-06-03 14:14:48 +01:00
Hieu Hoang
cfee8cde88 cube pruning - queue 2016-06-02 17:47:04 +01:00
Hieu Hoang
2f371b79b1 cube pruning 2016-06-02 16:21:15 +01:00
Hieu Hoang
c7e0221e6b cube pruning 2016-06-02 13:57:29 +01:00
Hieu Hoang
720af91638 cube pruning 2016-06-02 12:58:29 +01:00
Hieu Hoang
cc5631c469 replumb n-best arcs. Now works 2016-06-01 23:44:46 +01:00
Hieu Hoang
ffbd774afd debug 2016-06-01 23:30:26 +01:00
Hieu Hoang
14fd2e1def Merge ../mosesdecoder into perf_moses2 2016-06-01 21:24:38 +01:00
Hieu Hoang
2ad1eacef7 eclipse 2016-06-01 21:24:28 +01:00
Hieu Hoang
3b879e17e7 runs on 1 word sentence but crashes otherwise 2016-06-01 20:53:52 +01:00
Hieu Hoang
48bfb00248 change symbol bind pointer to object. May be source of data leak 2016-06-01 20:21:44 +01:00
Hieu Hoang
f7e5b0fe0d debug 2016-06-01 20:11:01 +01:00
Hieu Hoang
082b10fa11 debug 2016-06-01 19:28:49 +01:00
Hieu Hoang
38d6ab09ad debug 2016-06-01 18:47:57 +01:00
Hieu Hoang
a05dc31059 debug 2016-06-01 16:36:44 +01:00
Hieu Hoang
8ef36d2925 debug 2016-06-01 16:12:50 +01:00
Hieu Hoang
9f9cec80e3 debug 2016-06-01 15:59:35 +01:00
Hieu Hoang
7e5e7b8614 debug 2016-06-01 15:27:13 +01:00
Hieu Hoang
7913c5d09d debug 2016-06-01 14:29:48 +01:00
Hieu Hoang
f7e910b68a debug 2016-05-31 18:34:26 +01:00
Hieu Hoang
ce85cf2529 set score for SCFG model 2016-05-30 21:23:23 +02:00
Hieu Hoang
eca081976e set state 2016-05-30 19:47:19 +02:00
Hieu Hoang
11f39c2278 set scores for scfg 2016-05-30 19:42:28 +02:00
Hieu Hoang
28cbc08fcc change EvaluateWhenApplied() to more closely resemble moses 2016-05-30 19:32:56 +02:00
Hieu Hoang
dacdc28bf4 BlankState() take System arg, not Manager 2016-05-30 17:21:12 +02:00
Hieu Hoang
e1ebefd7c8 eclipse 2016-05-26 23:44:37 +01:00
Hieu Hoang
178f5a9f18 kenlm chart state 2016-05-26 21:11:56 +01:00
Hieu Hoang
da4bd6dd51 EvaluateWhenApplied() for SCFG 2016-05-26 20:32:42 +01:00
Hieu Hoang
3f73c77f58 kenlm EvaluateInIsolation() for SCFG 2016-05-26 15:11:23 +01:00
Hieu Hoang
28816cbd1f mem leak in target phrases 2016-05-26 14:46:27 +01:00
Hieu Hoang
843a3033eb mem leak in hypo 2016-05-26 12:26:48 +01:00
Hieu Hoang
9c1d258b62 mem leak in active chart 2016-05-26 12:21:37 +01:00
Hieu Hoang
8466d6aaf5 mem leak in InputPath 2016-05-26 11:42:00 +01:00
Hieu Hoang
aa865da027 mem leak 2016-05-25 22:25:10 +01:00