moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2025-01-02 17:09:36 +03:00
Code Issues Packages Projects Releases Wiki Activity
16,304 Commits 136 Branches 26 Tags 147 MiB
d9b2855cdd
Commit Graph

57 Commits

Author SHA1 Message Date
Phil Williams
01a9dd2305 extract-target-trees.py: support for new-style trace files 2015-08-14 16:53:24 +01:00
David Madl
3c30210dad Fix 'Use of uninitialized value' error through explicit setting of 0s in hash. 
Fixes the following errors in bootstrap-hypothesis-difference-significance.pl on Perl v5.14.2:

Use of uninitialized value $coocUpd in numeric gt (>) at /fs/lofn0/dmadl/software/mosesdecoder/scripts/analysis/bootstrap-hypothesis-difference-significance.pl line 317.
Use of uninitialized value $b in numeric lt (<) at /fs/lofn0/dmadl/software/mosesdecoder/scripts/analysis/bootstrap-hypothesis-difference-significance.pl line 543.
Use of uninitialized value $coocUpd in addition (+) at /fs/lofn0/dmadl/software/mosesdecoder/scripts/analysis/bootstrap-hypothesis-difference-significance.pl line 314.
Use of uninitialized value $coocUpd in numeric gt (>) at /fs/lofn0/dmadl/software/mosesdecoder/scripts/analysis/bootstrap-hypothesis-difference-significance.pl line 317.
Use of uninitialized value $a in numeric gt (>) at /fs/lofn0/dmadl/software/mosesdecoder/scripts/analysis/bootstrap-hypothesis-difference-significance.pl line 552.
 2015-07-14 13:05:22 +01:00
Jeroen Vermeulen
ef028446f3 Add license notices to scripts. 
This is not pleasant to read (and much, much less pleasant to write!) but
sort of necessary in an open project.  Right now it's quite hard to figure
out what is licensed how, which doesn't matter much to most people but can
suddenly become very important when people want to know what they're being
allowed to do.

I kept the notices as short as I could.  As far as I could see, everything
without a clear license notice is LGPL v2.1 or later.
 2015-05-29 18:30:26 +07:00
Jeroen Vermeulen
a25193cc5d Fix a lot of lint, mostly trailing whitespace. 
This is lint reported by the new lint-checking functionality in beautify.py.
(We can change to a different lint checker if we have a better one, but it
would probably still flag these same problems.)

Lint checking can help a lot, but only if we get the lint under control.
 2015-05-17 20:04:04 +07:00
Jeroen Vermeulen
9bdcb5f7c1 Fix more Python lint. 
This is about the last that isn't in contrib or generated files.  At this
point we can start doing regular lint checks, at least on the Python files,
without being completely inundated with warnings.
 2015-05-16 18:03:54 +07:00
Hieu Hoang
6162223690 add use warnings to all perl scripts 2015-04-13 20:42:33 +04:00
Hieu Hoang
2d1da3219d consistently use 'env perl' command for environments where the 1st perl in PATH isn't the default perl. Which is kinda stupid 2015-04-02 17:38:56 +04:00
Ondrej Bojar
01e364d1e6 use --n=0 to check coverage of full sents 2014-09-10 14:33:55 +02:00
Phil Williams
943f6a9901 extract-target-trees.py: update to match current trace format 2013-11-10 16:09:06 +00:00
Phil Williams
06081f7ddb extract-target-trees.py: minor fixes, code style 2012-12-17 18:49:50 +00:00
Barry Haddow
2b4e61d826 Merge branch 'trunk' into miramerge 
Compiles, not tested.

Conflicts:
	Jamroot
	OnDiskPt/PhraseNode.h
	OnDiskPt/TargetPhrase.cpp
	OnDiskPt/TargetPhrase.h
	OnDiskPt/TargetPhraseCollection.cpp
	mert/BleuScorer.cpp
	mert/Data.cpp
	mert/FeatureData.cpp
	moses-chart-cmd/src/Main.cpp
	moses/src/AlignmentInfo.h
	moses/src/ChartManager.cpp
	moses/src/LM/Ken.cpp
	moses/src/LM/Ken.h
	moses/src/LMList.h
	moses/src/LexicalReordering.h
	moses/src/PhraseDictionaryTree.h
	moses/src/ScoreIndexManager.h
	moses/src/StaticData.h
	moses/src/TargetPhrase.h
	moses/src/Word.cpp
	scripts/ems/experiment.meta
	scripts/ems/experiment.perl
	scripts/training/train-model.perl
 2012-07-17 13:36:50 +01:00
Eva Hasler
f592c222fe revert mode changes 2012-07-04 12:25:21 +01:00
Hieu Hoang
debe090426 Change Bin to RealBin. Thanks to Tom Hoar 2012-06-26 11:57:23 -04:00
Eva Hasler
6a6a35c65e fix start weights in experiment.perl, add hypothesis queue for picking hope and fear translations, add variations to 1slack formulation 2012-06-01 01:49:42 +01:00
Eva Hasler
30deedde9f changed permission, everything changed.. 2012-05-10 18:54:24 +01:00
hieuhoang1972
b88fad16f8 create valid html header, according to Tomas Hudik 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4336 1f5c12ca-751b-0410-a591-d2e778427230
 2011-10-12 10:18:36 +00:00
pjwilliams
a064f799e0 Add scripts/analysis/extract-target-trees.py 
Usage: extract-target-trees.py [FILE]

Reads moses-chart's -T output from FILE or standard input and writes trees to
standard output in Moses' XML tree format.

git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4233 1f5c12ca-751b-0410-a591-d2e778427230
 2011-09-19 09:08:24 +00:00
bojar
66b71a7f5c Ondrej's little tools to examine weight settings 
not quite fit for public use, esp. the -summarize.sh one...


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4071 1f5c12ca-751b-0410-a591-d2e778427230
 2011-07-08 00:11:10 +00:00
mphi
ddabdf6b1b added support for arbitrary encodings via the $IO_ENCODING global variable on line 23; set to UTF8 by default 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3739 1f5c12ca-751b-0410-a591-d2e778427230
 2010-11-29 09:04:44 +00:00
mphi
1f6e9b488b the script now calculates the p-value and confidence intervals not only using BLEU, but also the NIST score; 
improved confidence interval representation (avg+-stddev);

fixed bugs



git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3345 1f5c12ca-751b-0410-a591-d2e778427230
 2010-06-22 20:17:42 +00:00
bojar
21de1e121f Merge branch 'master' into moses-svn 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3192 1f5c12ca-751b-0410-a591-d2e778427230
 2010-04-21 14:48:32 +00:00
mphi
9e8352a041 modified the implementation, removing unnecessary repetition, thus making the whole process approximately fifty times faster 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2866 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-07 09:11:09 +00:00
bojar
ff05e5a1b5 list frequent mismatched tokenizations first 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2852 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-03 16:37:08 +00:00
bojar
9b10946f10 fixed regexes to read current -osg format 
verbose at bad lines


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2850 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-03 14:35:21 +00:00
bojar
594e5e8acd adding a handy script for suspicious tokenization 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2845 1f5c12ca-751b-0410-a591-d2e778427230
 2010-02-03 14:23:06 +00:00
bojar
55e3ee4a30 just setting the executable bit 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2795 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-29 19:49:37 +00:00
bojar
2097e45edd a handy script for calculating out-of-vocabulary rate of n-grams 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2794 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-29 19:48:29 +00:00
bojar
0889b9efff renaming .pl -> .perl 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2674 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-26 17:23:41 +00:00
bojar
0e26f91865 don't organize to stacks by default, accept --organize-to-stacks 
read from stdin as well


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2673 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-26 17:20:28 +00:00
bojar
536c7bdbcc commiting a script by Loic Barrault to display moses search graph 
(-output-search-graph) using graphviz dot


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2672 1f5c12ca-751b-0410-a591-d2e778427230
 2010-01-26 17:01:12 +00:00
mphi
17c3cfffac added unpaired significance evaluation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2328 1f5c12ca-751b-0410-a591-d2e778427230
 2009-05-12 18:56:01 +00:00
mphi
8a4c6a2c63 pus significance test into proper location 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1915 1f5c12ca-751b-0410-a591-d2e778427230
 2008-10-23 09:16:33 +00:00
hieuhoang1972
3c07c5df4d add svn id comments to start of file 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1307 1f5c12ca-751b-0410-a591-d2e778427230
 2007-03-14 22:22:36 +00:00
lexi_birch
93937b529d Making remaining scripts os independent re pawd/pwd 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1103 1f5c12ca-751b-0410-a591-d2e778427230
 2006-12-29 13:45:21 +00:00
hieuhoang1972
ddd2fdeb20 Fix automount partition bug 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1098 1f5c12ca-751b-0410-a591-d2e778427230
 2006-12-29 00:59:59 +00:00
bojar
c6c02a83c6 Just a short description added. 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@823 1f5c12ca-751b-0410-a591-d2e778427230
 2006-09-21 12:23:42 +00:00
eherbst
c646717009 trying to fix caching 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@775 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-17 12:31:52 +00:00
eherbst
9c7ffb1fbb thought I had added this before 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@772 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-16 16:38:19 +00:00
eherbst
24cd2f3441 updating docs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@771 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-16 16:37:11 +00:00
eherbst
674c609fcd adding show-phrases-used 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@768 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-16 14:51:04 +00:00
eherbst
c34aca3053 modified sentence-by-sentence to handle multiple outputs; 
edited cache handling in newsmtgui (should increase speed and decrease errors)


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@767 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-16 14:49:10 +00:00
eherbst
486f88157f add formatting for sentence strings to make token comparison more accurate 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@761 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-15 20:17:41 +00:00
eherbst
25767cd5b0 fixed background-color HTML 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@757 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-15 18:20:05 +00:00
eherbst
1374aefc6d - fixed caching behavior of Corpus to remove gibberish and cache everything 
- fixed javascript sorting in sentence-by-sentence


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@735 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-14 22:18:54 +00:00
eherbst
87056b15a7 added my script to the docs 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@724 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-14 16:13:29 +00:00
eherbst
20f49a1ded fixed legend display 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@723 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-14 16:09:21 +00:00
bojar
f246845489 utf8 output 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@686 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-13 02:44:28 +00:00
bojar
e6914693a1 reports also the top N words 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@668 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-11 21:48:39 +00:00
bojar
8f504a1d9b a handy script to count words that passed through the decoder unchanged (mostly because they're unknown); can exclude numbers and punctuation 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@667 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-11 21:26:24 +00:00
bojar
15566bb58a utf8, support for printing source, too 
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@618 1f5c12ca-751b-0410-a591-d2e778427230
 2006-08-10 14:35:09 +00:00