moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-12-29 15:04:05 +03:00
Code Issues Packages Projects Releases Wiki Activity
12,366 Commits 136 Branches 26 Tags 147 MiB
08be3b6b4f
Commit Graph

3673 Commits

Author SHA1 Message Date
Hieu Hoang
2830ed9968 use Nadir's backoff code 2014-05-12 15:40:18 +01:00
Hieu Hoang
1d401817a5 backoff fix for memory and on-disk phrase-tables 2014-05-12 15:16:11 +01:00
Hieu Hoang
44fec57c53 backoff fix 2014-05-12 14:06:47 +01:00
Hieu Hoang
f5851ecfbd Merge ../mosesdecoder into hieu 2014-05-08 21:44:26 +01:00
Hieu Hoang
6a80643ab6 minor compile error with clang 2014-05-08 21:44:06 +01:00
Hieu Hoang
7632fbd51c Merge ../mosesdecoder into hieu 2014-05-08 21:09:35 +01:00
Hieu Hoang
91afa4b4f1 delete chart translation option if -inf score 2014-05-08 18:48:10 +01:00
Hieu Hoang
b64d87f1ba Merge branch 'master' of github.com:moses-smt/mosesdecoder 2014-05-08 17:58:22 +01:00
Hieu Hoang
15566f7fc5 extend FF framework by add StackVec* argument to 2nd Evaluate(). NULL if phrase-based 2014-05-08 17:51:45 +01:00
Hieu Hoang
3ef5d76860 add MaxSpanFreeNonTermSource. Need to change FF framework to take in derivation 2014-05-08 16:34:04 +01:00
Barry Haddow
171e151705 Only surface factor in hypergraph 2014-05-08 16:31:29 +01:00
Hieu Hoang
594aee3322 Merge branch 'master' of ../mosesdecoder into hieu 2014-05-08 11:42:19 +01:00
Hieu Hoang
14d892a858 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2014-05-08 11:26:32 +01:00
Hieu Hoang
d12f91841a Segfault in nbest list creation. UTIL_THROW_IF is the opposite of assert 2014-05-08 11:26:17 +01:00
Hieu Hoang
814fb20aff 0 scores for FF SetSourcePhrase 2014-05-07 19:29:09 +01:00
Hieu Hoang
4e977f0f1f Merge ../mosesdecoder into hieu 2014-05-07 12:21:49 +01:00
Hieu Hoang
b436570ab0 delete unused files 2014-05-07 12:21:19 +01:00
Hieu Hoang
71242a3578 Merge ../mosesdecoder into hieu 2014-05-07 12:11:24 +01:00
Barry Haddow
5dbf47afd3 Want number of source words covered in hg output 2014-05-02 12:01:03 +01:00
Barry Haddow
10c4077ae9 Add source-span to hg output, comments 2014-05-01 21:37:07 +01:00
Hieu Hoang
2beed771d2 minor error. Segfault if phrase length is bigger than max-phrase-length 2014-05-01 16:47:17 +01:00
Hieu Hoang
d4b270e796 redo creation of input paths for lattices. Should redo OOV handling as phrase-table 2014-05-01 15:10:16 +01:00
Hieu Hoang
17c10d45f3 merge 2014-05-01 14:52:38 +01:00
Hieu Hoang
1465cf94b7 debugging info 2014-04-30 20:47:58 +01:00
Hieu Hoang
26abab8e5c debugging info 2014-04-30 20:20:07 +01:00
Ulrich Germann
ae4ab9aae0 Merge branch 'master' of https://github.com/moses-smt/mosesdecoder 2014-04-29 15:50:39 +01:00
Hieu Hoang
4c74a5d638 bug in lattice decoding. Some arcs can span more than 1 node 2014-04-29 14:41:20 +01:00
Ulrich Germann
07d8b66366 Merge branch 'master' of https://github.com/moses-smt/mosesdecoder 2014-04-29 11:01:46 +01:00
Hieu Hoang
6182750b70 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2014-04-28 14:29:49 +01:00
Hieu Hoang
9e33f37fb4 bug in lattice decoding. Some target phrase collection can have 0 size. Should execute OOV processing 2014-04-28 14:29:39 +01:00
Ulrich Germann
7c145d045b Merge branch 'master' into dynamic-phrase-tables 
Conflicts:
	contrib/server/Jamfile
	contrib/server/mosesserver.cpp
 2014-04-28 10:00:07 +01:00
Hieu Hoang
da6ade7d94 get-by-line-number.perl 2014-04-24 17:48:12 +01:00
Ulrich Germann
e5b739946f Merge branch 'master' of https://github.com/moses-smt/mosesdecoder 2014-04-23 18:29:39 +01:00
Hieu Hoang
d7380d6d9e don't add label to m_targetLabelSet if no hypotheses. Assert error in parsing 2014-04-23 17:11:25 +01:00
Hieu Hoang
2c14b506b4 merge problem 2014-04-23 16:11:09 +01:00
Hieu Hoang
6a9eb6c848 minor leak showing for lex reordering. Just refactor 2014-04-23 15:12:27 +01:00
Hieu Hoang
3f32e48f97 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2014-04-23 12:11:50 +01:00
Hieu Hoang
00505ba048 minor leak 2014-04-23 11:36:25 +01:00
Hieu Hoang
b917ac552f strange assert failure when stack is empty 2014-04-22 19:05:08 +01:00
Hieu Hoang
eed865bd35 Merge ../mosesdecoder into hieu 2014-04-21 23:05:25 +01:00
Phil Williams
ff8ac92be1 moses_chart: oops, don't output more distinct translations than requested 2014-04-21 14:46:22 +01:00
Phil Williams
b308bd5657 moses_chart: enable ChartKBestExtractor. 2014-04-21 10:19:00 +01:00
Hieu Hoang
51f020b2c4 Merge ../mosesdecoder into hieu 2014-04-20 00:20:25 +01:00
Phil Williams
568685cb66 ChartKBestExtractor: fix memory leak, clean-up code 2014-04-19 10:29:41 +01:00
Hieu Hoang
def22cef44 Merge branch 'hieu' of github.com:hieuhoang/mosesdecoder into hieu 2014-04-18 18:11:56 +01:00
Hieu Hoang
7f41ab2390 trying to count the total number of possible translations 2014-04-18 16:32:29 +01:00
Barry Haddow
7681cd4a15 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2014-04-17 20:50:27 +01:00
Hieu Hoang
8d69327eb1 merge 2014-04-17 20:22:40 +01:00
Hieu Hoang
a4d32a2b09 minor compile errors in ChartKBestExtractor 2014-04-17 20:04:02 +01:00
Barry Haddow
b9c25afd45 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2014-04-17 19:02:57 +01:00
Phil Williams
66d0fe81e2 moses_chart: add ChartKBestExtractor (not enabled yet) 
Implements algorithm 3 from Huang and Chiang (2005)
 2014-04-15 17:34:21 +01:00
Ulrich Germann
abfbd936d7 Merge branch 'dynamic-phrase-tables' of file:///fs/sif0/germann/git/mosesdecoder into dynamic-phrase-tables 2014-04-14 00:12:21 +01:00
Ulrich Germann
27d233796d Bug fix in Word::CreateFromString(...). 2014-04-12 22:10:40 +01:00
Hieu Hoang
75213181ae Merge branch 'master' of github.com:moses-smt/mosesdecoder 2014-04-12 17:20:49 +02:00
Ulrich Germann
f85bf72a8e Merge branch 'master' into dynamic-phrase-tables 
Conflicts:
	moses/TranslationModel/UG/mmsapt.cpp
 2014-04-11 19:24:37 +01:00
Ulrich Germann
b7f2e18efc Rewrite of Word::CreateFromString() to accommodate elimination of 
ConfusionNet::String2Word().
 2014-04-11 15:24:52 +01:00
Hieu Hoang
e347020049 add FF ReferenceComparison 2014-04-11 11:03:50 +01:00
Hieu Hoang
0ec2fe0169 add FF RuleAmbiguity 2014-04-11 10:42:38 +01:00
Hieu Hoang
1e116a21ae add FF RuleAmbiguity 2014-04-11 10:29:57 +01:00
Hieu Hoang
9644a30858 add FF RuleAmbiguity 2014-04-11 10:22:03 +01:00
Ulrich Germann
24545ba1de Changes due to new pure virtual function in PhraseTable. 2014-04-11 04:28:34 +01:00
Ulrich Germann
84a72198d1 Shortcut in Word::CreateFromString() didn't work. 2014-04-11 00:37:38 +01:00
Hieu Hoang
1686686e65 start on FF ReferenceComparison 2014-04-10 21:47:06 +01:00
Hieu Hoang
91d6bfe0d5 start on FF ReferenceComparison 2014-04-10 21:34:34 +01:00
Kenneth Heafield
40c819d285 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2014-04-09 10:03:02 -07:00
Kenneth Heafield
c61231fad9 Fix BackwardTest failure which was just compiler-drive differences in floating point precision. 
BOOST_CHECK_CLOSE's tolerance is proportional to the value on the left, so 0.0 has zero tolerance.  Changed to an explicit range.
 2014-04-09 10:01:19 -07:00
Hieu Hoang
65326f205f -stack 0 means no stack pruning 2014-04-09 10:33:33 +01:00
Hieu Hoang
0e98f1bc06 bug in ControlRecombinationState 2014-04-09 09:59:52 +01:00
Hieu Hoang
4c634023ce add FF CountNonTerms 2014-04-08 17:19:26 +01:00
Hieu Hoang
7ae44a96f1 add FF CountNonTerms 2014-04-08 17:16:24 +01:00
Ulrich Germann
21f141ce0d Merge branch 'master' into dynamic-phrase-tables 
Conflicts:
	moses/InputPath.cpp
	moses/Word.cpp
 2014-04-08 14:58:29 +01:00
Ulrich Germann
ad678ac0ec Replaced calls to String2Word by calls to Word::CreateFromString(). String2Word is deprecated (code duplication) and should be removed. Some reformatting for readability. 2014-04-08 12:44:21 +01:00
Ulrich Germann
43a51bb5d7 Some reformatting for readability. Special check for factor-less operation in Word::CreateFromString(). 2014-04-08 12:43:21 +01:00
Ulrich Germann
f82e79d237 Replaced calls to String2Word by calls to Word::CreateFromString(). String2Word is deprecated (code duplication) and should be removed. 2014-04-08 12:41:30 +01:00
Hieu Hoang
bf11ff6251 merge error 2014-04-07 14:07:16 +01:00
Hieu Hoang
6b1e147cff add -default-non-term-for-empty-range-only 2014-04-07 14:00:40 +01:00
Hieu Hoang
5d61639763 Merge branch 'hieu' of github.com:hieuhoang/mosesdecoder into hieu 2014-04-07 12:01:55 +01:00
Hieu Hoang
8b7e122f3d Merge ../mosesdecoder into hieu 2014-04-07 12:00:47 +01:00
Hieu Hoang
2ea2224ba1 output source side of rule, if available 2014-04-07 12:00:13 +01:00
Hieu Hoang
59e26ff54d output source side of rule, if available 2014-04-07 11:54:57 +01:00
Hieu Hoang
daf4e57ea5 no scores for FF SetSourcePhrase 2014-04-07 11:39:34 +01:00
Hieu Hoang
b98dac2971 merge 2014-04-06 19:07:13 +01:00
Hieu Hoang
09dc0e9913 add FF/SetSourcePhrase 2014-04-06 16:57:54 +01:00
Rico Sennrich
395285d879 more logging for incremental chart search 2014-04-04 15:54:48 +01:00
Ulrich Germann
8c04078d14 Reformatting for readability. 2014-04-03 20:33:40 +01:00
Hieu Hoang
6fa561e39e comment out debugging message 2014-04-03 18:24:48 +01:00
Hieu Hoang
67b1cbce13 error in constrained decoding 2014-04-03 18:03:58 +01:00
Hieu Hoang
5c633fcbf8 error in constrained decoding 2014-04-03 18:01:52 +01:00
Hieu Hoang
7e7cc64c1d debugging 2014-04-03 17:21:31 +01:00
Hieu Hoang
0953b5caca debugging 2014-04-03 16:54:46 +01:00
Rico Sennrich
1f435340f0 faster pruning in chart decoding 2014-03-26 11:23:59 +00:00
Ulrich Germann
f63606a92b 1. Renamed PhraseDictionary member function release() to Release(). 
2. Added option to check phrase prefixes for early stopping of
   InputPath generation when translating confusion networks.
 2014-03-25 01:49:24 +00:00
Ulrich Germann
80b031769d (1) Bug fixes related to locking. (2) pstats cache disabled when translating confusion networks. 2014-03-25 01:48:49 +00:00
Ulrich Germann
84299006da Added another constructor to TSA_tree_iterator. 2014-03-25 01:43:00 +00:00
Ulrich Germann
f5d969ab37 Added missing library for custom-pt.cc 2014-03-25 01:42:06 +00:00
Ulrich Germann
344b7a0759 Changed verbosity level for "read confusion net with format ..." to 2. 2014-03-25 01:41:21 +00:00
Hieu Hoang
5e5ad96729 segfault when there's nothing in the stack 2014-03-24 16:50:43 +00:00
Ulrich Germann
728ceb4e4c Added missing #include. 2014-03-24 14:51:51 +00:00
Ulrich Germann
14e52c9b94 Errors occurring when accessing data files now throw exceptions. 2014-03-24 13:39:15 +00:00
Ulrich Germann
ed825cd234 Bug fix: dependency .../UG/generic//generic was missing for custom-pt 2014-03-24 13:33:46 +00:00
Ulrich Germann
20aa9dd189 Adapted to configurable map type in pstats. 2014-03-24 13:32:41 +00:00
Ulrich Germann
794defa238 Numerous bug fixes; implemented caching and improved life cycle 
management for TargetPhraseCollections.
 2014-03-24 13:30:22 +00:00
Ulrich Germann
7fd87943ea Reinstated pstats caching. 2014-03-24 13:29:05 +00:00
germann
2063da274d Made member Tokenindex.ridx mutable. 2014-03-24 12:34:59 +00:00
germann
5ec4436b07 Initial check-in. 2014-03-24 12:33:17 +00:00
germann
c12c25770a Added virtual member function release(TargetPhraseCollection*) to 
PhraseDictionary, which gets called by the destructor of InputPath to
release TargetPhraseCollections that aren't needed any more.
 2014-03-24 12:29:46 +00:00
germann
042e840045 Added mmlib as a dependency for FF_Factory.o 2014-03-24 12:23:18 +00:00
germann
f7ec2ee09a 1. Fixed order of initializations in constructor so gcc won't complain. 
2. Added call  to PhraseDictionary.release(TargetPhraseCollection*) to
   release used TargetPhraseCollection's.
 2014-03-24 10:55:47 +00:00
germann
ec6fd21603 Make sure m_numInputScores is initialized properly. 2014-03-24 10:53:29 +00:00
Hieu Hoang
e5a6d63cf2 Merge branch 'master' of github.com:moses-smt/mosesdecoder into hieu 2014-03-21 21:41:04 +00:00
Ulrich Germann
1f32bd0d08 Fixed header inclusions. 2014-03-21 16:49:44 +00:00
Rico Sennrich
fb16df8c00 typo in last commit 2014-03-21 11:59:41 +00:00
Rico Sennrich
45630a5851 various optimizations to make CYK+ parser several times faster and eat less memory. 
speed-up of decoding depends on how much time is spent in parser:
10-50% speed-up for string-to-tree systems observed (more on long sentences and with high max-chart-span).

if you only use hiero or string-to-tree models (but none with source syntax), use compile-option --unlabelled-source for (small) efficiency gains.
 2014-03-21 11:12:24 +00:00
Ulrich Germann
4685aeadd2 Bug fix. 2014-03-21 11:11:24 +00:00
Ulrich Germann
d5c96eff35 Minor edit of a comment. 2014-03-21 11:08:37 +00:00
Ulrich Germann
f030986f51 Bug fixes. Added some sanity checks. 2014-03-21 11:07:17 +00:00
Ulrich Germann
43e6c68586 Changed default method for PhraseScoreFwd. Added minor debugging message. 2014-03-19 20:54:49 +00:00
Ulrich Germann
7e344d1c53 1. Added variant of CreateRuleLookupManager variant as required by the new 
pure virtual function in PhraseTable.h (from the master branch).

2. Bug fix in pool_stats.
 2014-03-19 20:43:47 +00:00
Ulrich Germann
c91fb5cc84 Added mmlex-lookup utility program. 2014-03-19 20:41:29 +00:00
Hieu Hoang
172a33c53c probing pt source vocab 2014-03-19 13:20:00 +00:00
Hieu Hoang
f4befcf125 probing pt source vocab 2014-03-19 12:59:55 +00:00
Hieu Hoang
49d3564a1b changes to probing pt v2 2014-03-19 11:39:27 +00:00
Ulrich Germann
ef974cd6ad Merge branch 'master' into dynamic-phrase-tables 2014-03-18 18:25:35 +00:00
Ulrich Germann
740c9dc096 Memory-mapped stuff is now compiled only with --with-mm. 2014-03-18 18:23:59 +00:00
Ulrich Germann
ef2c012470 Merge branch 'master' into dynamic-phrase-tables 
Conflicts:
	moses/Jamfile
 2014-03-18 18:23:23 +00:00
Ulrich Germann
febb2afc4d Initial check-in. 2014-03-18 12:23:53 +00:00
Ulrich Germann
f20220066b Added choice of denominator for PhraseScoreFwd. 2014-03-18 12:23:31 +00:00
Ulrich Germann
a8eb6645c7 Bug fix. Added sanity check when adding data to dynamic suffix array. 2014-03-18 12:22:29 +00:00
Ulrich Germann
0562415ac0 Added program calc-coverage. 2014-03-18 12:21:12 +00:00
Ulrich Germann
e55dfa26b3 Added program calc-coverage. 2014-03-18 12:20:55 +00:00
Ulrich Germann
4aa88aaf2c Bug fix in constructor. 2014-03-18 12:20:29 +00:00
Ulrich Germann
a11b79175b Added function fill_token_seq. 2014-03-18 12:19:35 +00:00
Ulrich Germann
3f9cefe44e Removed some debugging messages. 
Moved fill_token_seq to tpt_tokenindex.h.
 2014-03-18 12:18:05 +00:00
Hieu Hoang
8fd0dff972 Merge branch 'master' of github.com:moses-smt/mosesdecoder into hieu 2014-03-18 08:30:31 +00:00
Kenneth Heafield
fc61084743 Compiler warning 2014-03-17 07:18:11 -07:00
Phil Williams
69f92bd6e4 moses_chart: implement -output-unknowns option. 2014-03-17 13:09:54 +00:00
Hieu Hoang
bff071fc80 Merge branch 'master' of github.com:moses-smt/mosesdecoder into hieu 2014-03-15 22:22:55 +00:00
Phil Williams
04dbd3c7aa moses_chart: more efficient scope-3 parsing if sentence length < max-chart-span 2014-03-14 08:49:09 +00:00
Ulrich Germann
e0f95fee06 Bug fixes in dynamic phrase tables. 2014-03-14 02:42:38 +00:00
Hieu Hoang
044ff868fc merge 2014-03-13 21:36:41 +00:00
Ulrich Germann
ce75b58f6f Routine check-in. 2014-03-13 13:41:32 +00:00
Ulrich Germann
94657fd589 Work in progress. 2014-03-12 23:13:44 +00:00
Ulrich Germann
9e104ed6a1 Reverted to version prior to last check-in. 2014-03-12 19:18:25 +00:00
Ulrich Germann
240f52255b Disabled 'rt' in the header specifications. Didn't work in this branch. 2014-03-12 08:09:01 +00:00
Ulrich Germann
9025ac065f Added utilities: 
- mam2symal converts memory-mapped word alignments to symal format
- mam_verify performs a sanity check on memory-mapped word alignments
 2014-03-12 08:06:55 +00:00
Ulrich Germann
b26eabba39 Merge branch 'master' of https://github.com/moses-smt/mosesdecoder 
Conflicts:
	moses/Jamfile
 2014-03-11 22:29:55 +00:00
Kenneth Heafield
8ae4d153c8 Consolidated, consistent rt target fixes single-threaded build 2014-03-11 11:29:40 -07:00
Ulrich Germann
c02fbf7664 Completely rewritten. Now multi-threaded. 2014-03-11 13:57:42 +00:00
Ulrich Germann
76eb3d56b9 Proper initialization of class members of InputFeature. 2014-03-11 13:56:40 +00:00
Ulrich Germann
d93c93df87 Reformatting for readability. 2014-03-11 13:55:16 +00:00
Ulrich Germann
a7c85780ee Merge branch 'master' into dynamic-phrase-tables 
Conflicts:
	phrase-extract/score-main.cpp
 2014-03-10 14:25:45 +00:00
Ulrich Germann
fdc504d47a Changes on main branch files while I was working on dynamic phrase tables. 2014-03-10 14:08:00 +00:00
Ulrich Germann
6e4035fb12 Changes on main branch files while I was working on dynamic phrase tables. 2014-03-10 14:06:10 +00:00
Ulrich Germann
aa8ba7d9a7 Put alignment functionality into a separate class. Not working yet --- work in progress! 2014-03-10 12:03:27 +00:00
Ulrich Germann
ff4ce426e7 Made scorer in PScoreLex public for development purposes. Reset default number of workers to 20. 2014-03-10 12:02:05 +00:00
Ulrich Germann
f7ee316e12 Added initialization of wlex21 and COOCraw during loading. 2014-03-10 11:59:58 +00:00
Ulrich Germann
aad5d67947 Added option to also count raw cooccurrences. 2014-03-10 11:58:46 +00:00
Ulrich Germann
9cf86f6191 Added class Alignment as a friend and wlex21 and COOCraw for development purposes while working on word alignment issues. 2014-03-10 11:57:40 +00:00
Ulrich Germann
9159729ad0 Made internal table COOC public for development purposes. 2014-03-10 11:56:22 +00:00
Hieu Hoang
5e93c3f804 conflicts 2014-03-07 12:18:18 +00:00
Ulrich Germann
81ed9937e1 Routine check-in. 2014-03-05 11:53:05 +00:00
Ulrich Germann
2b19b71095 Routine check-in. 2014-03-04 15:51:59 +00:00
Rico Sennrich
c27ecbe5c6 rename SyntaxConstraintFeature to TreeStructureFeature 
(makes it clearer what it does; build/print the internal tree structure in string-to-tree decoding)

no longer rely on name of FF for printing internal trees
 2014-03-03 13:56:19 +00:00
Ulrich Germann
6c37b8d252 Routine check-in. 2014-03-03 12:13:41 +00:00
Hieu Hoang
b521bad865 Merge branch 'master' of https://github.com/moses-smt/mosesdecoder into hieu 2014-03-01 12:56:29 +00:00
Jun-ya NORIMATSU
14d9d50b76 Merge remote-tracking branch 'origin/master' 2014-02-28 13:39:16 +09:00
Jun-ya NORIMATSU
19bc1c317d improve DALMwrapper again. 2014-02-28 13:38:35 +09:00
Hieu Hoang
1e5cd16fbf xml parsing for source syntax 2014-02-26 18:09:27 +00:00
Hieu Hoang
b48f410bee don't check object pointer before deleting /Markus Elfring 2014-02-26 10:43:37 +00:00
Hieu Hoang
c48d8b5e31 update last weight override 2014-02-26 10:32:48 +00:00
Ulrich Germann
2b181ee691 Fixed Mmsapt constructor. 2014-02-25 03:10:16 +00:00
Ulrich Germann
4c003edb0d Fixed #include-s. 2014-02-25 03:09:19 +00:00
Hieu Hoang
b9c6c7a2a6 need to zero out all weights, especially sparse weights 2014-02-23 21:35:34 +00:00
Hieu Hoang
7509e8bbaa reset sparse weights 2014-02-23 20:46:16 +00:00
Hieu Hoang
1716514eda create ResetWeights(). Only for dense weights 2014-02-23 20:07:28 +00:00
Ulrich Germann
a8d66cd68d Removed Mmsapt constuctor with both descriptor and config line. 2014-02-22 00:27:07 +00:00
Ulrich Germann
817e3695e0 Fixed some include paths. 2014-02-22 00:25:58 +00:00
Ulrich Germann
1252700c44 Removed constructor with both description and config line. 2014-02-22 00:25:02 +00:00
Ulrich Germann
4b95c3a906 Merge branch 'dynamic-phrase-tables' of ssh://thor//home/germann/git/mosesdecoder into dynamic-phrase-tables 
due to resetting the location of the remote repository.
 2014-02-21 01:09:38 +00:00
Ulrich Germann
ac238ef2d7 Changed construction from a given token sequence to allow partial matches. 2014-02-20 23:56:11 +00:00
Ulrich Germann
8afe62145b Minor fix to make the compiler stop complain about unused typedef. 2014-02-20 23:54:15 +00:00
Ulrich Germann
e1d07e7475 Added pid2str conversion method to convert from phrase ids to the string. 2014-02-20 23:53:15 +00:00
Ulrich Germann
9536cf49e9 Phrase look-up now also gathers phrase orientation info (work in progress). 2014-02-20 23:51:17 +00:00
Ulrich Germann
6c66b9c631 Added Jamfile to produce try-align 2014-02-20 23:50:07 +00:00
Ulrich Germann
683635ce25 Minor fix to make the compiler stop complaining about unused variables. 2014-02-20 23:48:56 +00:00
Ulrich Germann
061b861639 Small test program for phrase-based alignment via mmsapt. 2014-02-20 23:29:37 +00:00
Ulrich Germann
c259e10b23 Various changes. 2014-02-20 23:28:01 +00:00
Ulrich Germann
9bcc315644 Added phrase-based word alignment to mmsapt (work in progress!). 2014-02-20 23:25:36 +00:00
Jun-ya NORIMATSU
007799eb07 modify DALMwrapper. 2014-02-19 17:49:42 +09:00
Hieu Hoang
5155fa26ce Merge branch 'master' into hieu 2014-02-17 22:36:24 +00:00
Jun-ya NORIMATSU
ba63f1eb97 improve DALMwrapper. 2014-02-18 00:00:28 +09:00
Jun-ya NORIMATSU
4488d97629 tune DALMWrapper. 2014-02-14 20:22:53 +09:00
Barry Haddow
b0b90dd498 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2014-02-13 21:38:23 +00:00
Hieu Hoang
50cadc754f use boost::unordered_map for CacheColl. Marginally faster 2014-02-11 03:43:58 +00:00
Ulrich Germann
a6ce081e15 Minor changes. 2014-02-08 18:25:46 +00:00
Ulrich Germann
6d2818fa57 Fixed include path for msapt.h 2014-02-08 18:22:33 +00:00
Ulrich Germann
11aa1422a5 Added a few mm-related things. 2014-02-08 18:07:16 +00:00
Ulrich Germann
594272ce05 Changed function count_tokens so that it can be run without passing a filter explicitly. 2014-02-08 18:06:11 +00:00
Ulrich Germann
9899364c46 Added implicit add-1 smoothing. 2014-02-08 18:03:18 +00:00
Ulrich Germann
40fbe226e4 Added private members numSent and numWords. 2014-02-08 18:02:03 +00:00
Ulrich Germann
66822b279b Added append function to grow imTtracks dynamically in a thread-safe fashion. 2014-02-08 18:00:27 +00:00
Ulrich Germann
9f317f4849 Minor fix. 2014-02-08 17:58:05 +00:00
Ulrich Germann
5f8ae20d01 Added dynamicly updatable corpus; updated or added query functions. 2014-02-08 17:56:48 +00:00
Ulrich Germann
784654c831 Initial check-in. 2014-02-08 17:50:26 +00:00
Ulrich Germann
584626a767 Added a few programs. 2014-02-08 17:49:28 +00:00
Ulrich Germann
5c131f196c Minor changes. 2014-02-08 17:22:57 +00:00
Ulrich Germann
4fb00ea6fd Minor fixes. 2014-02-08 16:55:05 +00:00
Ulrich Germann
0702926dff Added special copy constructor that adds new sentences to the new imTSA. 2014-02-08 16:53:15 +00:00
Ulrich Germann
e81e1772f8 Added capability to add sentence pairs to imBitext. Various minor fixes. 2014-02-08 16:48:39 +00:00
Jun-ya NORIMATSU
d78778d1fc prepare for the next version of DALM. 2014-02-07 15:04:35 +09:00
Barry Haddow
ffd7af1168 Fixes to hypergraph and htk outputs, better cmd line help. 
Mixing boost paths and strings is bad.
Leaks in htk output.
assert that should be an exception
 2014-02-06 20:15:25 +00:00
Hieu Hoang
31857c47a2 Merge branch 'master' into nikolay_probingPT 2014-02-05 17:14:51 +00:00
Hieu Hoang
0efd37e147 Merge branch 'master' of github.com:moses-smt/mosesdecoder into hieu 2014-02-04 09:01:56 +00:00
Matthias Huck
86ee3e15a4 new version of the score tool 
which is now capable of dealing with additional properties in an appropriate manner
 2014-01-29 18:37:42 +00:00
Hieu Hoang
b34b4b978f different max-spans depending on whether source non-term is X or syntactic 2014-01-27 17:53:53 +00:00
Hieu Hoang
cb2805151b delete -adjacent-only. Rubbish 2014-01-27 12:25:02 +00:00
Hieu Hoang
b68a906fdd output list of available feature functions, separated by space, rather than lines 2014-01-25 10:58:00 +00:00
Hieu Hoang
9745924937 output list of available feature functions, separated by space, rather than lines 2014-01-25 10:39:47 +00:00
Hieu Hoang
38d106bd56 everything works. Tested with en-cs. V. slow 2014-01-23 18:32:24 +00:00
Hieu Hoang
1a74c80cdb everything should work. Not yest tested 2014-01-23 14:24:45 +00:00
Hieu Hoang
1aad3dfdce moved 2014-01-23 13:07:33 +00:00
Hieu Hoang
6c3b8587ba bjam 2014-01-23 00:51:03 +00:00
Hieu Hoang
5800ff848c bjam 2014-01-22 20:57:06 +00:00
Hieu Hoang
52ab05a154 set scores and alignments (almost) 2014-01-22 18:41:58 +00:00
Hieu Hoang
e505a462a7 probing hash pt compiles. Need to hook up vocab 2014-01-22 18:24:51 +00:00
Hieu Hoang
63b9eac201 integration of Nikolay's probing pt into Moses 2014-01-22 13:06:10 +00:00
Hieu Hoang
39858ce1ff leak 2014-01-21 18:07:12 +00:00
Hieu Hoang
fcadf4511a leak 2014-01-21 17:11:16 +00:00
Hieu Hoang
8fe07d1c9c oops. Minor compile error 2014-01-20 17:01:53 +00:00
Hieu Hoang
839d1c6978 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2014-01-20 16:26:45 +00:00
Hieu Hoang
b448e77639 register feature function HyperParameterAsWeight 2014-01-20 16:12:46 +00:00
Rico Sennrich
c8108ca71d some sparse feature functions couldn't be set to be non-tuneable in moses.ini 2014-01-20 15:56:36 +00:00
Rico Sennrich
5d2b4d8efc sparse feature weights can now be set in moses.ini; overrides weights in weight-file. 2014-01-20 15:54:17 +00:00
Hieu Hoang
20e7d078a6 add feature function HyperParameterAsWeight 2014-01-17 17:59:36 +00:00
Rico Sennrich
742e59f1e0 minor optimization (minimize performance impact of SoftMatchingFeature code if disabled) 2014-01-17 11:57:52 +00:00
Rico Sennrich
9e177cb472 SyntaxConstraintFeature (without any actual constraints; useful to build/output syntax tree from GHKM tree fragments) 2014-01-16 18:45:26 +00:00
Rico Sennrich
ed25bb2b99 soft matching of target-side nonterminals 2014-01-16 18:34:33 +00:00
jiejiang
c55161dff4 merge c11 revert with mingw changes 2014-01-16 01:08:01 +00:00
jiejiang
8486d30d74 Merge remote-tracking branch 'origin/master' 2014-01-16 00:08:41 +00:00
jiejiang
3c11a72c93 c11 macro 2014-01-16 00:06:58 +00:00
jiejiang
5f1217d793 merged upstream with origin for mingw 2014-01-15 18:16:56 +00:00
Rico Sennrich
6d6fca340c Revert "C++11 test; put in the middle of things to make sure people complain if their compiler doesn't support it" 
This reverts commit 32ccbef3b9.
 2014-01-15 16:16:03 +00:00
jiejiang
ead60c4c89 exclude all changes for bjam on mingw, which is buggy on linux 2014-01-16 00:00:33 +08:00
Rico Sennrich
df30085bbe pass regtest with C++11 and gcc 4.7 2014-01-15 09:27:20 +00:00
Rico Sennrich
32ccbef3b9 C++11 test; put in the middle of things to make sure people complain if their compiler doesn't support it 2014-01-14 17:15:25 +00:00
Hieu Hoang
fbe1332bc7 SkeletonPT uses phrase-table cache to store trarget phrases, instead of it's own collection 2014-01-14 11:08:57 +00:00
Hieu Hoang
fccf2ea43a get rid of all aborts from decoder 2014-01-13 18:32:22 +00:00
Hieu Hoang
b5558c36b8 get rid of all aborts 2014-01-13 14:37:05 +00:00
Hieu Hoang
07902ca550 minor memleak 2014-01-12 20:42:46 +00:00
Hieu Hoang
372359ba81 minor memleak 2014-01-12 18:56:43 +00:00