moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-09-19 15:17:10 +03:00
Code Issues Packages Projects Releases Wiki Activity
13,916 Commits 136 Branches 26 Tags 147 MiB
315610c02a
Commit Graph

4563 Commits

Author SHA1 Message Date
Ulrich Germann
b6c887b370 Minor bug fix in logging biased sampling for phrase lookup. 2015-04-03 20:18:55 +01:00
Ulrich Germann
93ce2423df 1. A context string for biased sampling in Mmsapt can now be provided on the 
command line with --context-string. Not available in server mode yet.
2. Numerous bug fixes related to biased sampling.
3. Biased sampling now checks that the sampling sticks to the bias. If
   the distribution of samples deviates too much from the bias, samples
   whose selection would push the sample distribution even further from the bias
   are not considered, even if that means that fewer samples are chosen in total.
 2015-04-03 16:16:52 +01:00
Jeroen Vermeulen
c52d7be025 Make an include path explicit. 
Better to see clearly where that header is supposed to come from, since
it lives halfway across the codebase.

I think this may also make the "classifier" dance in moses/Jamfile
unnecessary, which would be a nice simplification, but I'm not
confident enough to just make the change.
 2015-04-03 17:47:48 +07:00
Jeroen Vermeulen
ebc0930500 Replace use of tmpnam with boost::filesystem. 
Silences a few annoying warnings from gcc: "tmpnam is dangerous" (and
the suggestion to use mkstemp instead).
 2015-04-02 10:42:06 +07:00
XapaJIaMnu
29a729c99b Remove old obsolete probingPT tests 2015-04-01 16:58:21 +01:00
Hieu Hoang
1a61e1405f delete External FF. FF framework changes too fast to be able to keep this up-to-date 2015-03-31 19:45:59 +04:00
Ulrich Germann
a9dbced81d Bug fix. 2015-03-30 02:56:49 +01:00
Ulrich Germann
fcbfc5a535 Feature functions and the constructors of TranslationOptionCollections 
now have access to the current translation task.

This was done to allow context-sensitive processing (if provided by the FF).
 2015-03-30 01:20:17 +01:00
Ulrich Germann
79cd40d2c4 Disabled temporarily. Needs to be adapted to API changes in Mmsapt. 2015-03-29 23:58:17 +01:00
Ulrich Germann
2899645992 Cleanup. 2015-03-29 23:57:14 +01:00
Ulrich Germann
3541838a46 Included TargetPhraseCollectionCache.* in fakelib mmsapt. 2015-03-29 23:55:47 +01:00
Ulrich Germann
76b76e4db6 Bug fixes. 2015-03-29 23:51:51 +01:00
Ulrich Germann
18d2547497 Added member for storing annotations by feature functions. 2015-03-29 23:51:22 +01:00
Ulrich Germann
1525f1ea62 Cleanup. 2015-03-29 23:44:06 +01:00
Ulrich Germann
529a766da7 Initial check-in. 2015-03-29 23:43:50 +01:00
Jeroen Vermeulen
b124d99330 Use boost::filesystem for "rm -rf". 
Replaces a system() call (which was a portability problem) and fixes,
en passant, a warning about its return value being ignored.
 2015-03-29 18:33:58 +07:00
Jeroen Vermeulen
789a2e2bc3 Fix some compile warnings (gcc 4.9.2). 
Mostly signed/unsigned comparisons and reordered member
initializations; also a few unused variables.

There are more, but if I chip away at them for a while, who knows, it
may catch on and warnings may eventually become socially stigmatizing.
:)
 2015-03-29 18:10:51 +07:00
Ulrich Germann
a706569844 Thread-safe classes for storing context-specific information. 2015-03-28 14:44:40 +00:00
Ulrich Germann
1b23edf62f Cache for the N most recently used TargetPhraseCollections. Refactored out of mmsapt.h. 2015-03-28 14:41:08 +00:00
Jeroen Vermeulen
a9c8f44896 Modernize "C" includes in moses. 
This is one of those little chores in managing a long-lived C++
project: standard C headers like stdio.h and math.h now have their own
place in the C++ standard as resp. cstdio, cmath, and so on.  In this
branch the #include names are updated for the moses/ subdirectory; more
branches to follow.

C++11 adds cstdint, but to support compilation with the previous
standard, that change is left for later.
 2015-03-28 20:09:03 +07:00
Ulrich Germann
c7ad2571f5 Making GetSource() in Manager and Basemanager virtual breaks one regression tests. 2015-03-26 23:38:58 +00:00
Ulrich Germann
e827896027 Harmonized spelling of CleanUpAfterSentenceProcessing(...). 2015-03-26 23:37:29 +00:00
Ulrich Germann
9dc75bfd8a Managers and feature functions now have access to the entire TranslationTask, not just the InputType. 2015-03-26 18:25:54 +00:00
Ulrich Germann
b3ff4ff13a Managers have now access to the entire translation task. 2015-03-26 16:37:04 +00:00
Ulrich Germann
d364211d90 More bjam fiddling. 2015-03-26 12:21:55 +00:00
Hieu Hoang
1064aaacbe delete typedefs for UINT32 and UINT64. MSVC now has uint32_t and uint64_t /Ken 2015-03-25 00:55:39 +00:00
Ulrich Germann
ddf7bc3e23 1. Added FeatureFunction::Setup(TranslationTask const&) const to allow FFs to set themselves up for specific input. 
2. FeatureFunction::ChangeSource should not be allow to change the pointer to the source.
 2015-03-21 17:09:41 +00:00
Ulrich Germann
8ca11d941d 1. Lifetime of tasks in ThreadPool is now managed via shared pointers. 
2. Code cleanup in IOWrapper and a bit elsewhere.
 2015-03-21 16:12:52 +00:00
Ulrich Germann
85d2567b57 Renamed MosesServer::TranslationTask to MosesServer::TranslationRequest and made that a subtype of Moses::TranslationTask to avoid name confusion. 2015-03-21 15:56:20 +00:00
Ulrich Germann
ee4e396a4d Removed pointer to TranslationTask in InputTypes again. Not the right place to store this information. 2015-03-21 15:29:37 +00:00
Ulrich Germann
fd56ba6afb Renamed moses/server/TranslationTask.* to moses/server/TranslationRequest.* to avoid naming conflicts. 2015-03-21 14:00:30 +00:00
Ulrich Germann
6c73693c49 Library path for xmlrpc-c given via --with-xmlprc-c was not included in the compile requirements. 2015-03-20 22:32:42 +00:00
Ulrich Germann
b1c9d8a528 Bjam is the most awful build system in the world. 2015-03-20 18:51:49 +00:00
Ulrich Germann
e40fbbc2cb Added pragma to tell me if Moses is built with server functionality at compilation time. 2015-03-20 18:45:46 +00:00
Ulrich Germann
9c19ad25b0 Changed order of #includes because of compiler warnings. 2015-03-20 18:43:29 +00:00
Ulrich Germann
386db6529a More fiddling with bjam. 2015-03-19 14:36:43 +00:00
Ulrich Germann
d1cb14edc0 Fixed Jamfiles. 2015-03-19 13:45:58 +00:00
Ulrich Germann
6f52340c2b Merge branch 'master' of https://github.com/moses-smt/mosesdecoder 2015-03-18 21:04:18 +00:00
Rico Sennrich
eab513b635 relational dependency language model 2015-03-18 17:39:45 +00:00
Rico Sennrich
1568afb737 on-the-fly unbinarization of internal tree structure (for translation models extracted from binarized treebanks) 2015-03-18 17:36:32 +00:00
Ulrich Germann
286800abba Addresses issues with compiling server capabilities into the Moses main executable. 2015-03-18 13:50:30 +00:00
Ulrich Germann
058abc32c3 Fixes (?) numerous bugs discovered when the code was actually compiled (an error in the bjam configuration prevented this). 2015-03-18 13:46:15 +00:00
Hieu Hoang
63d8b390b4 Changes to RUleScope from private branch. More codelite projects 2015-03-17 11:50:33 +00:00
Hieu Hoang
25feb7e47b option to change the estimated score only, not actuall score 2015-03-17 10:25:34 +00:00
Ulrich Germann
dcffbb5f4d Made LRModel::ReorderingType an enumerated type. 2015-03-16 00:24:11 +00:00
Ulrich Germann
085c88cc7b Eliminated sources of some compiler warnings (unused variables; signed/usigned comparisons). 2015-03-15 22:45:01 +00:00
Ulrich Germann
ad805c133b Instances of InputType (and derived classes) now know which TranslationTask (if any) created them. 
This is a first step towards providing phrase tables etc. access to context information etc.
associated with specific translation tasks.
 2015-03-15 20:38:31 +00:00
Ulrich Germann
2a66a55c85 Added document map (maps from sentences to document ids) to Bitext class. 
Minor overhaul to the bias regime, which allows to specify bias by document
name (as provided in the document map) rather than by sentence in the static
parallel corpus.
 2015-03-15 13:32:09 +00:00
Ulrich Germann
bc91743820 Merge branch 'master' of https://github.com/moses-smt/mosesdecoder 2015-03-11 23:32:12 +00:00
Ulrich Germann
a49b76be3f Quick hack to make moses not stumble over double-dash parameter specifications. 2015-03-11 23:32:06 +00:00
Kenneth Heafield
54304fd473 Merge branch 'master' of https://github.com/moses-smt/mosesdecoder 
Conflicts:
	moses/ExportInterface.cpp
 2015-03-11 17:43:48 -04:00
James Zhang
23704613de added a simple translation interface 2015-03-11 17:33:17 -04:00
Phil Williams
e8a7163f0d Add SyntaxInputWeight feature function 
Currently only used for forest input.
 2015-03-10 11:07:04 +00:00
Ulrich Germann
137b07a486 Merge branch 'master' of https://github.com/moses-smt/mosesdecoder 2015-03-10 10:41:47 +00:00
Ulrich Germann
51824355f9 Sampling now keeps track of counts for hierarchical lexicalized reordering. 2015-03-10 10:41:41 +00:00
Matthias Huck
524ed4406e pragma once 2015-03-09 21:44:54 +00:00
Ulrich Germann
ccf4cb838c Merge branch 'master' of https://github.com/moses-smt/mosesdecoder 2015-03-09 01:27:02 +00:00
Ulrich Germann
524376fad4 Code cleanup. 2015-03-09 00:34:47 +00:00
Ulrich Germann
c4885d87c0 Code cleanup 2015-03-09 00:34:18 +00:00
Ulrich Germann
c1d2313a66 Moved caching of lexical reordering scores from class TranslationOption to class TargetPhrase. 
This was done so that phrase tables can add this information (if available) as extra annotation
to TargetPhrases, in preparation of providing lexical reordering models with sampling phrase tables.
 2015-03-09 00:30:01 +00:00
Ulrich Germann
ddea89312e Commented out an #include that triggered a 'deprecated header' warning. 2015-03-09 00:08:31 +00:00
Hieu Hoang
afe8b4162e link with openmp for srilm 2015-03-08 21:25:24 +00:00
Hieu Hoang
c08ebc9dae link with openmp for srilm 2015-03-08 21:23:13 +00:00
Ales Tamchyna
9266d65304 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2015-03-06 13:44:46 +01:00
Ales Tamchyna
9224779695 include scores of source-context features in topt collection verbose output 2015-03-06 13:44:38 +01:00
Rico Sennrich
b3d2b11a36 make BilingualLM play nice with new chart decoding score deltas 2015-03-06 08:52:13 +00:00
Ulrich Germann
612e014e79 Organized command line options thematically. 
This is the first step in migration to the use of boost::program_options for GNU/Linux-style handling of command line options.
 2015-03-05 00:34:02 +00:00
Ales Tamchyna
6058bfc61b bugfix, BP only applies when candidate is shorter than ref :) 2015-03-04 18:02:50 +01:00
Ales Tamchyna
ff1cae919b minor refactoring in VW feature 2015-03-04 17:40:05 +01:00
Ales Tamchyna
34649b74d3 implement BLEU2+1 as training loss for VW 2015-03-04 17:10:59 +01:00
Ales Tamchyna
7601618477 some refactoring of VW, towards customizable loss calculation in training 2015-03-04 14:26:26 +01:00
Ulrich Germann
4ad0db1d16 Merge branch 'master' of https://github.com/moses-smt/mosesdecoder 2015-03-04 00:20:14 +00:00
Ulrich Germann
982286b38f Folded contrib/server/mosesserver.cpp into the moses executable. 
With immediate effect, mosesserver is deprecated, use moses --server instead.
 2015-03-04 00:07:11 +00:00
Matthias Huck
2543c78b71 RulePairUnlexicalizedSource feature function 2015-03-02 14:32:18 +00:00
Ulrich Germann
9291bb80ca Merge branch 'master' of https://github.com/moses-smt/mosesdecoder 2015-03-02 11:24:30 +00:00
Ulrich Germann
e1c02ee57c Code reorganization. 2015-03-02 11:23:00 +00:00
Hieu Hoang
0829e5196d path arg /Cheng Yong 2015-02-28 13:59:36 +00:00
Matthias Huck
0d4678577b floor value, glue rules, some cleanup 2015-02-27 02:39:43 +00:00
Matthias Huck
0afc261251 SoftSourceSyntacticConstraintsFeature: Sparse label pair scores 2015-02-26 20:27:02 +00:00
Matthias Huck
342abcf8fc Model1Feature: score caching for improved efficiency 2015-02-25 23:13:15 +00:00
Matthias Huck
0a6adcde3a Model1Feature: special UNK token in MGIZA vocabularies 2015-02-25 01:27:08 +00:00
Matthias Huck
39c1ef52dc Model1Feature: unnecessary includes & special UNK token in MGIZA vocabularies 2015-02-25 00:52:15 +00:00
Matthias Huck
3c8d48f8af Model1Feature: a simple IBM Model 1 scorer, 
source-to-target with global source-sentence context
 2015-02-25 00:11:31 +00:00
Hieu Hoang
28fbf07c37 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2015-02-24 12:35:12 +00:00
Hieu Hoang
59986d8c81 change default mac-chart-span 10->20. /Arefeh Kazemi 2015-02-24 12:35:00 +00:00
Phil Williams
0bff50449e Fix -search-algorithm 8 2015-02-23 11:12:00 +00:00
Matthias Huck
372d14a9e8 missing condition 2015-02-20 21:46:38 +00:00
Matthias Huck
f317a2cebd PhraseOrientationFeature 
better efficiency via precomputation of non-terminal orientation classes,
plus a bugfix (weights vector was not read correctly)
 2015-02-20 20:53:50 +00:00
Matthias Huck
9f4ebd83e0 TargetPhrase: can store arbitrary data 2015-02-20 20:53:25 +00:00
Hieu Hoang
32de075022 beautify 2015-02-19 12:27:23 +00:00
Hieu Hoang
6d4bad0f78 Merge branch 'master' of github.com:moses-smt/mosesdecoder 2015-02-18 21:56:48 +00:00
Hieu Hoang
add14db330 use VERBOSE instead of cerr 2015-02-18 21:56:35 +00:00
Phil Williams
357f33b0d5 Fix -output-unknowns for -search-algorithm 3 (moses_chart) 2015-02-18 16:22:13 +00:00
Hieu Hoang
b223256040 park Joint LM. Doesn't work anymore 2015-02-18 15:11:03 +00:00
Matthias Huck
6028c7cf9c tuneable-components config parameter for feature functions 2015-02-16 18:39:25 +00:00
Ales Tamchyna
f6884c55a1 restoring broken evaluation of translation option list 2015-02-16 12:32:00 +01:00
Ales Tamchyna
a46daa1938 target (internal) bigrams for VW 2015-02-16 11:36:16 +01:00
Ales Tamchyna
b05fc4c9bc properly register bigram VW feature 2015-02-16 11:30:36 +01:00
Ales Tamchyna
a9926e7f49 source bigram feature for VW 2015-02-16 10:59:55 +01:00
Ulrich Germann
064b01c054 Code reorganization and cleanup around Lexical Reordering Round 2. 
Hope it compiles this time.
 2015-02-15 16:34:54 +00:00