moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2025-01-09 04:56:57 +03:00
Code Issues Packages Projects Releases Wiki Activity
15,915 Commits 136 Branches 26 Tags 147 MiB
0d50bd6f2b
Commit Graph

1529 Commits

Author SHA1 Message Date
Hieu Hoang
ceaa21ab37 eclipse 2016-01-18 23:59:48 +00:00
Hieu Hoang
98fe921443 synchronise with old cold 2016-01-18 21:17:20 +00:00
Hieu Hoang
f470b90645 eclipse 2016-01-18 21:13:46 +00:00
Hieu Hoang
8a276e2a71 Jamfile cleanup 2016-01-18 15:37:04 +00:00
Hieu Hoang
b2bffd1754 move moses2 into root directory 2016-01-18 15:00:49 +00:00
Hieu Hoang
1de37e1f08 add CreateProbingPT2 2016-01-18 14:41:22 +00:00
Hieu Hoang
31bab49e16 add CreateProbingPT2 2016-01-18 12:26:37 +00:00
Hieu Hoang
9239658c51 local variable 2016-01-16 00:31:33 +00:00
Hieu Hoang
8583e1dd92 debugging cache 2016-01-15 19:09:17 -05:00
Hieu Hoang
096c994708 query cache 2016-01-15 23:58:28 +00:00
Hieu Hoang
9f652dd0fd return CreateTargetPhraseStruct 2016-01-15 23:30:52 +00:00
Hieu Hoang
d92ec391c1 return CreateTargetPhraseStruct 2016-01-15 22:55:18 +00:00
Hieu Hoang
759c9163aa return CreateTargetPhraseStruct 2016-01-15 22:52:09 +00:00
Hieu Hoang
ec0278f390 don't check prefix 2016-01-15 22:20:16 +00:00
Hieu Hoang
41d4300f25 get ready to use cache 2016-01-15 19:56:54 +00:00
Hieu Hoang
2c1011a304 cache 2016-01-15 18:18:33 +00:00
Hieu Hoang
5c386896d7 don't implement caching in pt base class. Up to each pt 2016-01-15 17:01:32 +00:00
Hieu Hoang
2d709d92c9 Get ready to cache 2016-01-15 12:21:42 +00:00
Hieu Hoang
9c2f32c57d Get ready to cache 2016-01-15 11:36:07 +00:00
Hieu Hoang
604643da5d factor out hash key for source phrase. Get ready to cache 2016-01-15 11:28:45 +00:00
Hieu Hoang
c131e3eb73 save most frequent source phrase to cache file 2016-01-14 16:00:10 +00:00
Hieu Hoang
0482500482 variable rename 2016-01-14 12:49:14 +00:00
Hieu Hoang
4da2364473 recycle bitmap 2016-01-14 12:26:10 +00:00
Hieu Hoang
14c5ac92c6 don't init bitmap in constructor. Get ready to recycle bitmap 2016-01-14 12:19:46 +00:00
Hieu Hoang
747349b2ad bitmap in mempool 2016-01-14 11:49:42 +00:00
Hieu Hoang
0f7ce2e871 bjam 2016-01-14 00:37:06 +00:00
Hieu Hoang
7ff5ca45bd check prefix 2016-01-13 22:00:28 +00:00
Hieu Hoang
7fa16da878 input path created in mempool 2016-01-13 21:47:16 +00:00
Hieu Hoang
2db2a4a797 use pointers to InputPath 2016-01-13 17:18:40 +00:00
Hieu Hoang
1fe4500505 get ready to check prefixes 2016-01-13 16:46:23 +00:00
Hieu Hoang
38f999fa3f Merge ../mosesdecoder into perf_moses2 2016-01-13 14:57:20 +00:00
Hieu Hoang
e0fb456dbb eclipse 2016-01-13 14:56:45 +00:00
Hieu Hoang
74285a4db3 support --cube-pruning-lazy-scoring 2016-01-12 15:47:30 +00:00
Hieu Hoang
aea7841756 support --cube-pruning-diversity 2016-01-12 15:12:28 +00:00
Hieu Hoang
00f8c119f1 don't copy from vector to Hypotheses 2016-01-12 14:13:19 +00:00
Hieu Hoang
61349d0b92 sort hypos 2016-01-12 13:34:15 +00:00
Hieu Hoang
85b6462a07 get ready to sort hypos 2016-01-12 12:53:24 +00:00
Hieu Hoang
24f00d7369 default value 2016-01-12 12:27:40 +00:00
Hieu Hoang
45f59cf973 use std::vector 2016-01-12 12:10:56 +00:00
Hieu Hoang
822a3afe09 get ready to sort hypos 2016-01-12 11:40:28 +00:00
Hieu Hoang
c63e678988 add NSCubePruningCardinalStack 2016-01-12 11:25:16 +00:00
Hieu Hoang
42e303982c add CubePruningCardinalStack 2016-01-11 17:51:14 +00:00
Hieu Hoang
582b6f7f2e comment out debugging message 2016-01-11 16:57:30 +00:00
Hieu Hoang
191ca0c002 separate queue for each stack, not each bitmap 2016-01-11 16:08:32 +00:00
Hieu Hoang
9bfb39d22f completed NSCubePruningPerBitmap 2016-01-11 15:47:53 +00:00
Hieu Hoang
421e7c909c add NSCubePruningPerBitmap 2016-01-11 14:57:32 +00:00
Hieu Hoang
fe6e04c80f comment out debug 2016-01-10 13:21:41 +00:00
Hieu Hoang
c701497c97 leak 2016-01-08 17:47:13 +00:00
Hieu Hoang
4bb3413ec7 queue is per Ministack, not per stack 2016-01-08 17:07:52 +00:00
Hieu Hoang
004dcd52f8 move Container() to Util2.h 2016-01-08 15:03:40 +00:00
Hieu Hoang
a3fa3ff82a Merge branch 'perf_moses2' of github.com:hieuhoang/mosesdecoder into perf_moses2 2016-01-08 14:44:00 +00:00
Hieu Hoang
3840c307fd DebugCounts() 2016-01-08 09:43:47 -05:00
Hieu Hoang
1eb3ff9d93 DebugCounts() for cube pruning 2016-01-08 14:43:14 +00:00
Hieu Hoang
51d9b4d102 DebugCounts() 2016-01-08 14:14:48 +00:00
Hieu Hoang
2cab9734be create cube pruning datastructures 1st, including ministacks 2016-01-08 13:42:16 +00:00
Hieu Hoang
361d831939 add Misc to NSCubePruningPerMiniStack 2016-01-08 12:34:35 +00:00
Hieu Hoang
8a1ee99e7a move Hypotheses to Stack.h 2016-01-08 11:50:33 +00:00
Hieu Hoang
9f6f510c45 re-added Stacks 2016-01-08 00:15:44 +00:00
Hieu Hoang
8aa298efa2 re-added Stacks 2016-01-07 19:04:22 +00:00
Hieu Hoang
d8dfda7de9 added NSCubePruningPerMiniStack 2016-01-07 18:43:55 +00:00
Hieu Hoang
0b7b8f583e eclipse 2016-01-07 12:23:34 +00:00
Hieu Hoang
5d1bc4b406 log prob 2016-01-07 12:06:49 +00:00
Hieu Hoang
bef1910c5e start log prob in binary pt 2016-01-07 10:46:54 +00:00
Hieu Hoang
08bbf273ea recycle mini stack 2016-01-06 21:27:41 +00:00
Hieu Hoang
de8b99ccc0 slight difference with Moses1 for 1st hypo 2016-01-06 17:42:23 +00:00
Hieu Hoang
4443a34052 support -tt 2016-01-06 16:35:24 +00:00
Hieu Hoang
4791e75a6b support -report-segmentation 2016-01-06 16:09:08 +00:00
Hieu Hoang
f66885166e delete Stacks class 2016-01-06 15:04:09 +00:00
Hieu Hoang
d8f65ecbc9 don't have vector of stacks. Just 1 stack 2016-01-06 14:54:40 +00:00
Hieu Hoang
5bbc0bf97d only require 1 stack. Reuse each time 2016-01-06 14:41:14 +00:00
Hieu Hoang
fbba014c8a static methods QueueItem::Create(). Recycle existing item, use 1 from recycle bin, or create new item 2016-01-06 12:13:22 +00:00
Hieu Hoang
0ee1f452db comments 2016-01-06 11:41:42 +00:00
Hieu Hoang
197fd1b903 use system pool for hypo 2016-01-05 16:58:27 +00:00
Hieu Hoang
749c531996 use system pool for hypo 2016-01-05 14:34:59 +00:00
Hieu Hoang
aebc27316d rename methods in Recyler 2016-01-05 11:53:17 +00:00
Hieu Hoang
7a92e78e3e get ready to change pool for hypo 2016-01-05 11:22:53 +00:00
Hieu Hoang
f179072902 cpu affinity offset 2016-01-04 00:19:35 +00:00
Hieu Hoang
abf41d04e9 cpu affinity offset 2016-01-03 23:22:50 +00:00
Hieu Hoang
4208115174 add Recycler class 2016-01-02 23:03:08 +00:00
Hieu Hoang
6994fc9a4b reference, not copy 2016-01-02 22:43:17 +00:00
Hieu Hoang
0c22176c66 thread-safe system pool 2016-01-02 21:44:44 +00:00
Hieu Hoang
3249d4a634 seen position set and get at the same time 2016-01-02 01:20:51 +00:00
Hieu Hoang
9bc7dd6157 cleanup 2016-01-02 00:46:48 +00:00
Hieu Hoang
42e8aadbd4 QueueItem recycler 2016-01-02 00:22:03 +00:00
Hieu Hoang
9b2cdac051 reuse QueueItem 2016-01-02 00:14:08 +00:00
Hieu Hoang
aae48b2859 add unused hypos from queue into recycler 2016-01-01 23:59:06 +00:00
Hieu Hoang
e869392051 reuse QueueItem 2016-01-01 23:45:27 +00:00
Hieu Hoang
d26f7d3a27 use pointers 2016-01-01 23:35:21 +00:00
Hieu Hoang
f73b90516a delete Recycler 2016-01-01 21:24:12 +00:00
Hieu Hoang
bce8c63f7b get ready to delet Recycler 2016-01-01 21:08:58 +00:00
Hieu Hoang
48a3a4260b leak 2016-01-01 19:10:05 +00:00
Hieu Hoang
4b0508a20b Revert "recycle query_result" 
This reverts commit b7d78d7a8b.
 2016-01-01 13:34:25 -05:00
Hieu Hoang
dcf486308e Revert "recycle query_result" 
This reverts commit aaa88d5d05.
 2016-01-01 13:34:24 -05:00
Hieu Hoang
785cc722fd Revert "leak" 
This reverts commit 4feb3e355c.
 2016-01-01 13:34:22 -05:00
Hieu Hoang
4feb3e355c leak 2016-01-01 18:07:57 +00:00
Hieu Hoang
aaa88d5d05 recycle query_result 2016-01-01 18:02:12 +00:00
Hieu Hoang
b7d78d7a8b recycle query_result 2016-01-01 17:58:19 +00:00
Hieu Hoang
e1ed7ee219 recycle vector in vbyte_decode_line() 2016-01-01 17:25:23 +00:00
Hieu Hoang
4af443d8af get ready to recycle more objects 2016-01-01 17:14:40 +00:00
Hieu Hoang
566faa7039 class scope recycler 2016-01-01 16:38:18 +00:00
Hieu Hoang
e097b4cf52 reset target_text when taken from recycler 2016-01-01 16:30:45 +00:00
Hieu Hoang
155a03df15 recycle target_text 2016-01-01 16:20:37 +00:00
Hieu Hoang
8107bcf1ea store pointer to vocab id 2016-01-01 15:12:16 +00:00
Hieu Hoang
0dec823c60 Revert "reuse query_result" 
This reverts commit 32f988be87.
 2016-01-01 08:49:57 -05:00
Hieu Hoang
32f988be87 reuse query_result 2016-01-01 13:41:44 +00:00
Hieu Hoang
7d1aeaa601 warnings 2015-12-31 17:28:11 +00:00
Hieu Hoang
6435f15a14 delete unused variable 2015-12-31 17:10:30 +00:00
Hieu Hoang
e26081e5c5 Revert "get ready to cache MiniStack" 
This reverts commit acc0fdf807.
 2015-12-31 11:42:12 -05:00
Hieu Hoang
8873d71497 Revert "cache MiniStack" 
This reverts commit b887885a04.
 2015-12-31 11:42:09 -05:00
Hieu Hoang
b887885a04 cache MiniStack 2015-12-31 16:38:18 +00:00
Hieu Hoang
acc0fdf807 get ready to cache MiniStack 2015-12-31 16:22:10 +00:00
Hieu Hoang
d39c7626f5 comment out prefetch 2015-12-31 10:01:17 -05:00
Hieu Hoang
6417924de0 use hypothesis::Prefetch() 2015-12-31 14:53:16 +00:00
Hieu Hoang
f1580def18 set methods in range class 2015-12-31 14:31:49 +00:00
Hieu Hoang
7d035c235d comment out prefetch 2015-12-31 09:26:09 -05:00
Hieu Hoang
adc38bc834 prefetch losing hypo for reuse 2015-12-31 13:56:28 +00:00
Hieu Hoang
d79d227418 return asap 2015-12-31 13:48:34 +00:00
Hieu Hoang
0915dbd472 force new hypo into place of losing hypo 2015-12-31 13:41:48 +00:00
Hieu Hoang
8f33d13ab2 move Add() into MiniStack 2015-12-31 13:30:34 +00:00
Hieu Hoang
6da80a599d output warning if no translation 2015-12-31 12:59:33 +00:00
Hieu Hoang
fbc2c60b36 no output to cerr 2015-12-31 12:56:47 +00:00
Hieu Hoang
bb60b02619 more mempool 2015-12-31 02:09:57 +00:00
Hieu Hoang
646079b8ec more mempool 2015-12-31 01:49:53 +00:00
Hieu Hoang
d1cceb1d38 more mempool 2015-12-31 01:28:21 +00:00
Hieu Hoang
844e8a16c4 anonymous objects 2015-12-31 00:58:48 +00:00
Hieu Hoang
d0951cde0a allocator for stack 2015-12-31 00:48:16 +00:00
Hieu Hoang
f4a07ef1fa allocator for stack 2015-12-31 00:29:13 +00:00
Hieu Hoang
00952efdec anonymous objects 2015-12-30 23:35:03 +00:00
Hieu Hoang
ff09020526 don't use pointers 2015-12-30 23:27:41 +00:00
Hieu Hoang
95ea1d8485 use my allocator 2015-12-30 23:15:28 +00:00
Hieu Hoang
7d4432a2d6 clear container using old method 2015-12-30 23:10:13 +00:00
Hieu Hoang
181914b822 use boost fast pool allocator 2015-12-30 22:58:12 +00:00
Hieu Hoang
08f61d69e9 use boost pool allocator 2015-12-30 22:51:50 +00:00
Hieu Hoang
95e59ac36f use pointers for mempool 2015-12-30 21:54:03 +00:00
Hieu Hoang
5f4b34b95f use pointers for mempool 2015-12-30 21:49:46 +00:00
Hieu Hoang
bc2f065f45 Mempool with Queue 2015-12-30 21:42:21 +00:00
Hieu Hoang
34946bc931 undo stack diversity 2015-12-30 19:20:00 +00:00
Hieu Hoang
599bc4855a create hypo from every edge. Increase diversity 2015-12-30 19:00:27 +00:00
Hieu Hoang
c5826035c2 Mempool with Queue 2015-12-30 16:46:42 +00:00
Hieu Hoang
448d0a3e9c Mempool with seen position 2015-12-30 16:33:48 +00:00
Hieu Hoang
654f2616e6 get ready to use Mempool with seen position 2015-12-30 16:16:36 +00:00
Hieu Hoang
2cdbad5f2d Revert "delete score breakdown pointer" 
This reverts commit 2f7e1b599b.
 2015-12-30 16:01:49 +00:00
Hieu Hoang
2f7e1b599b delete score breakdown pointer 2015-12-30 15:31:16 +00:00
Hieu Hoang
9cf5d24ef4 get ready to delete score breakdown pointer 2015-12-30 15:24:01 +00:00
Hieu Hoang
fbbd118b3e use only 1 score for estimated score, don't need Score object 2015-12-30 14:57:02 +00:00
Hieu Hoang
4e03fa5fba use only 1 score for estimated score, don't need Score object 2015-12-30 14:34:24 +00:00
Hieu Hoang
e7a5b2bafc use dequeue for recycler 2015-12-30 14:02:02 +00:00
Hieu Hoang
b2b5e5cf85 Revert "recycler unused bitmaps" 
This reverts commit 48b26851e0.
 2015-12-30 13:48:21 +00:00
Hieu Hoang
48b26851e0 recycler unused bitmaps 2015-12-30 13:40:52 +00:00
Hieu Hoang
e7247534ce Revert "use mempool in bitmap" 
This reverts commit 1bf356ee8d.
 2015-12-30 08:16:24 -05:00