moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2025-01-06 03:33:37 +03:00
Code Issues Packages Projects Releases Wiki Activity
2,452 Commits 136 Branches 26 Tags 147 MiB
6b153c67f8
Commit Graph

2452 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
jhclark
a4fa26aced And detect number of cores while we're at it 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4269 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-26 15:00:14 +00:00
jhclark
39747881cb Give more helpful error and status messages in regenerate-makefiles.sh 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4268 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-26 14:56:47 +00:00
nicolabertoldi
c89f11a29d pointer to the required release of IRSTLM 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4267 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-26 07:37:39 +00:00
heafield
12e15abe8a Fix file naming bug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4266 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-25 22:36:03 +00:00
chesio
e402ac25c5 Eppex is now properly released + reinjected into train-model.perl params. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4265 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-25 12:06:15 +00:00
hieuhoang1972
9762a65bd8 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4264 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-25 10:44:47 +00:00
heafield
8edf53dcf3 StringPiece. Does a body good. And avoids string copying. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4263 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-24 15:58:23 +00:00
heafield
55e24da4d5 Sync with changes made for cdec. Documentation. Remove an unnecessary copy. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4262 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-24 15:24:33 +00:00
heafield
d40ba48da3 Replace hypothesis ids with pointers to ChartHypothesis. This avoids locking. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4261 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-24 14:48:52 +00:00
pjwilliams
efddd0f390 Remove race condition from ChartHypothesis ID allocation in moses_chart. 
Thanks to Kenneth Heafield for pointing that out.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4259 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-24 10:06:21 +00:00
heafield
19879a8b1e -threads all option 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4252 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-22 22:29:56 +00:00
heafield
ddbfee788c Sorry about the compiler error. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4251 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-22 08:55:54 +00:00
hieuhoang1972
d2245390e0 visual studio build 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4250 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-22 05:39:32 +00:00
heafield
daf0e37175 Barry wanted to insure Factor is non-copyable. But STL containers want it to be copyable. Here's some friend magic to meet both demands. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4249 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-21 20:35:12 +00:00
heafield
72bb0e51ac Add file used by a test. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4248 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-21 20:15:03 +00:00
heafield
3402bdfe7a Merge mtm_lm into trunk. 
There's a fair number of files with no change that somebody must have touched in the branch so metadata is being recorded. 
Updates kenlm binary file format, sorry. 
It looks like OOV isn't being computed in EvaluateChart anyway, just phrasal.  
  


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4247 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-21 16:06:48 +00:00
hieuhoang1972
08805d6a9c comment revert 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4246 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-21 11:04:48 +00:00
hieuhoang1972
659e34735d xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4245 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-21 11:01:19 +00:00
heafield
fdc6c37ca4 Store the string in the factor itself. Use MurmurHash because it beats Boost's hash. Also, apparently nobody calls Exists. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4244 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-21 10:26:04 +00:00
heafield
4c8552b16f Cruft removal 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4243 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-20 19:29:24 +00:00
heafield
b9e433977d Redo FactorCollection including better locking 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4242 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-20 19:08:42 +00:00
heafield
ed30a602af Remove deprecated, and uncalled LoadVocab function 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4241 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-20 17:12:27 +00:00
nicolabertoldi
c2de897cd2 added support for a new class-based lm type provided by IRSTLM 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4240 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-20 14:46:50 +00:00
pjwilliams
a064f799e0 Add scripts/analysis/extract-target-trees.py 
Usage: extract-target-trees.py [FILE]

Reads moses-chart's -T output from FILE or standard input and writes trees to
standard output in Moses' XML tree format.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4233 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-19 09:08:24 +00:00
bhaddow
2cf26266f7 debug should be off by default 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4232 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-18 21:23:32 +00:00
hieuhoang1972
4313e335b5 print out span widths of non-terms. Extra argument --OutputNTLengths 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4230 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-16 17:13:34 +00:00
bhaddow
4d5b17f444 Option to create extract file with sentence ids 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4229 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-16 15:37:02 +00:00
phkoehn
7927439ce5 option to get unpruned search graph for chart decoder 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4228 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-16 11:58:53 +00:00
phkoehn
0ab0df5aac Hopkins&May Interpolation, bugfix for sparse features 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4227 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-16 11:55:49 +00:00
bhaddow
fc695c38a7 Implementation of sharding and resampling in mert. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4226 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-15 17:45:35 +00:00
hieuhoang1972
1e1eb4d29e print out span widths of non-terms. Extra argument --OutputNTLengths 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4225 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-14 18:16:05 +00:00
hieuhoang1972
149208ecba print out span widths of non-terms. Extra argument --OutputNTLengths 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4224 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-14 10:23:14 +00:00
hieuhoang1972
d68274d217 print out span widths of non-terms. Extra argument --OutputNTLengths 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4223 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-14 07:15:36 +00:00
hieuhoang1972
b1ca5e1fc8 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4222 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-13 19:01:04 +00:00
hieuhoang1972
b8606b3e70 xcode 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4221 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-13 18:51:28 +00:00
hieuhoang1972
358720c24a don't score with good-turing. phi has moved this calc to another file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4220 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-13 18:14:14 +00:00
bhaddow
d53350dd95 Regression test for lm oov feature 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4218 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-12 20:53:22 +00:00
leven101
3824ca3ea2 fixed operator [] adding bug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4216 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-12 18:04:56 +00:00
bhaddow
e95c004145 Implement LM OOV feature 
- still have to run regression
 - doesn't work for moses_chart (awaiting H & K LM state merge)


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4211 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-09 18:03:00 +00:00
hieuhoang1972
661544109c fix n-best label for word penalty 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4209 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-09 15:27:54 +00:00
servan
c55b03dd66 M TerScorer.cpp 
M    TerScorer.h


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4206 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-09 13:26:45 +00:00
bhaddow
0955474dea remove old tests 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4195 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-08 18:04:13 +00:00
bhaddow
8d558d2e04 Fix test script 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4194 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-08 18:03:30 +00:00
bhaddow
ed8d85acc1 moving mert tests 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4193 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-08 17:51:09 +00:00
bhaddow
2c585ce6e7 restore 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4186 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-07 16:42:46 +00:00
bhaddow
de51b69d03 remove (temporarily) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4185 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-07 16:40:55 +00:00
phkoehn
41a1849437 support for sparse feature functions (mert support only when using PRO) 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4184 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-07 16:37:33 +00:00
bhaddow
9fee4a97f2 clean-up 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4183 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-07 08:36:08 +00:00
bhaddow
ca5c0f19b7 Multi-threading of mert, for random restarts. 
Fix mert tests.


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4182 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-07 08:08:35 +00:00
bhaddow
37ebe7a62e Build common files into archive so we don't have to compile them 3 times 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4179 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-06 12:29:18 +00:00