moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-11-10 10:59:21 +03:00
Code Issues Packages Projects Releases Wiki Activity
9,292 Commits 136 Branches 26 Tags 147 MiB
9b8254a90e
Commit Graph

47 Commits

Author SHA1 Message Date
Barry Haddow
2b4e61d826 Merge branch 'trunk' into miramerge 
Compiles, not tested.

Conflicts:
	Jamroot
	OnDiskPt/PhraseNode.h
	OnDiskPt/TargetPhrase.cpp
	OnDiskPt/TargetPhrase.h
	OnDiskPt/TargetPhraseCollection.cpp
	mert/BleuScorer.cpp
	mert/Data.cpp
	mert/FeatureData.cpp
	moses-chart-cmd/src/Main.cpp
	moses/src/AlignmentInfo.h
	moses/src/ChartManager.cpp
	moses/src/LM/Ken.cpp
	moses/src/LM/Ken.h
	moses/src/LMList.h
	moses/src/LexicalReordering.h
	moses/src/PhraseDictionaryTree.h
	moses/src/ScoreIndexManager.h
	moses/src/StaticData.h
	moses/src/TargetPhrase.h
	moses/src/Word.cpp
	scripts/ems/experiment.meta
	scripts/ems/experiment.perl
	scripts/training/train-model.perl
 2012-07-17 13:36:50 +01:00
Eva Hasler
f592c222fe revert mode changes 2012-07-04 12:25:21 +01:00
Hieu Hoang
debe090426 Change Bin to RealBin. Thanks to Tom Hoar 2012-06-26 11:57:23 -04:00
Eva Hasler
6a6a35c65e fix start weights in experiment.perl, add hypothesis queue for picking hope and fear translations, add variations to 1slack formulation 2012-06-01 01:49:42 +01:00
Eva Hasler
30deedde9f changed permission, everything changed.. 2012-05-10 18:54:24 +01:00
hieuhoang1972
b88fad16f8 create valid html header, according to Tomas Hudik 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4336 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-12 10:18:36 +00:00
pjwilliams
a064f799e0 Add scripts/analysis/extract-target-trees.py 
Usage: extract-target-trees.py [FILE]

Reads moses-chart's -T output from FILE or standard input and writes trees to
standard output in Moses' XML tree format.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4233 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-19 09:08:24 +00:00
bojar
66b71a7f5c Ondrej's little tools to examine weight settings 
not quite fit for public use, esp. the -summarize.sh one...


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4071 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-08 00:11:10 +00:00
mphi
ddabdf6b1b added support for arbitrary encodings via the $IO_ENCODING global variable on line 23; set to UTF8 by default 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3739 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-29 09:04:44 +00:00
mphi
1f6e9b488b the script now calculates the p-value and confidence intervals not only using BLEU, but also the NIST score; 
improved confidence interval representation (avg+-stddev);

fixed bugs



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3345 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-22 20:17:42 +00:00
bojar
21de1e121f Merge branch 'master' into moses-svn 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3192 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-21 14:48:32 +00:00
mphi
9e8352a041 modified the implementation, removing unnecessary repetition, thus making the whole process approximately fifty times faster 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2866 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-07 09:11:09 +00:00
bojar
ff05e5a1b5 list frequent mismatched tokenizations first 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2852 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-03 16:37:08 +00:00
bojar
9b10946f10 fixed regexes to read current -osg format 
verbose at bad lines


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2850 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-03 14:35:21 +00:00
bojar
594e5e8acd adding a handy script for suspicious tokenization 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2845 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-03 14:23:06 +00:00
bojar
55e3ee4a30 just setting the executable bit 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2795 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-29 19:49:37 +00:00
bojar
2097e45edd a handy script for calculating out-of-vocabulary rate of n-grams 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2794 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-29 19:48:29 +00:00
bojar
0889b9efff renaming .pl -> .perl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2674 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-26 17:23:41 +00:00
bojar
0e26f91865 don't organize to stacks by default, accept --organize-to-stacks 
read from stdin as well


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2673 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-26 17:20:28 +00:00
bojar
536c7bdbcc commiting a script by Loic Barrault to display moses search graph 
(-output-search-graph) using graphviz dot


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2672 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-26 17:01:12 +00:00
mphi
17c3cfffac added unpaired significance evaluation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2328 1f5c12ca-751b-0410-a591-d2e778427230
 2009-05-12 18:56:01 +00:00
mphi
8a4c6a2c63 pus significance test into proper location 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1915 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-23 09:16:33 +00:00
hieuhoang1972
3c07c5df4d add svn id comments to start of file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1307 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-14 22:22:36 +00:00
lexi_birch
93937b529d Making remaining scripts os independent re pawd/pwd 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1103 1f5c12ca-751b-0410-a591-d2e778427230
 2006-12-29 13:45:21 +00:00
hieuhoang1972
ddd2fdeb20 Fix automount partition bug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1098 1f5c12ca-751b-0410-a591-d2e778427230
 2006-12-29 00:59:59 +00:00
bojar
c6c02a83c6 Just a short description added. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@823 1f5c12ca-751b-0410-a591-d2e778427230
 2006-09-21 12:23:42 +00:00
eherbst
c646717009 trying to fix caching 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@775 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-17 12:31:52 +00:00
eherbst
9c7ffb1fbb thought I had added this before 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@772 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-16 16:38:19 +00:00
eherbst
24cd2f3441 updating docs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@771 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-16 16:37:11 +00:00
eherbst
674c609fcd adding show-phrases-used 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@768 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-16 14:51:04 +00:00
eherbst
c34aca3053 modified sentence-by-sentence to handle multiple outputs; 
edited cache handling in newsmtgui (should increase speed and decrease errors)


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@767 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-16 14:49:10 +00:00
eherbst
486f88157f add formatting for sentence strings to make token comparison more accurate 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@761 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-15 20:17:41 +00:00
eherbst
25767cd5b0 fixed background-color HTML 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@757 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-15 18:20:05 +00:00
eherbst
1374aefc6d - fixed caching behavior of Corpus to remove gibberish and cache everything 
- fixed javascript sorting in sentence-by-sentence


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@735 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-14 22:18:54 +00:00
eherbst
87056b15a7 added my script to the docs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@724 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-14 16:13:29 +00:00
eherbst
20f49a1ded fixed legend display 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@723 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-14 16:09:21 +00:00
bojar
f246845489 utf8 output 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@686 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-13 02:44:28 +00:00
bojar
e6914693a1 reports also the top N words 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@668 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-11 21:48:39 +00:00
bojar
8f504a1d9b a handy script to count words that passed through the decoder unchanged (mostly because they're unknown); can exclude numbers and punctuation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@667 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-11 21:26:24 +00:00
bojar
15566bb58a utf8, support for printing source, too 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@618 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-10 14:35:09 +00:00
eherbst
cf8c271469 minor, and moved stuff around 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@588 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-08 23:38:45 +00:00
eherbst
384f8ccb07 adding sentence-by-sentence.pl: display all sentences in a corpus, system output vs. reference 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@552 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-08 00:09:40 +00:00
eherbst
8420ecf516 added statistical testing, both to compare different outputs and to get a confidence measure for a single output 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@529 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-07 17:22:39 +00:00
eherbst
54ab89deab seems this script does not have the same functionality as Ondrej's, and his are meant for training and this for analysis 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@430 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-31 22:14:08 +00:00
eherbst
3b46c17ace believe Ondrej has a script w/same functionality; will investigate 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@428 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-31 22:07:34 +00:00
eherbst
5cce8336c0 add CGI-based tool for calculating and displaying various error measures 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@427 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-31 22:05:11 +00:00
bojar
32853150fc added a placeholder 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@408 1f5c12ca-751b-0410-a591-d2e778427230
 2006-07-31 16:39:33 +00:00