Commit Graph

582 Commits

Author SHA1 Message Date
hieuhoang1972
15536efa03 took out FactorType
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@450 1f5c12ca-751b-0410-a591-d2e778427230
2006-08-01 20:02:39 +00:00
zens
ff15ed0732 - remove some obsolete code
- optional future costs for distortion model


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@444 1f5c12ca-751b-0410-a591-d2e778427230
2006-08-01 14:14:39 +00:00
hieuhoang1972
6bdf7eba18 memory pool cleanup
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@440 1f5c12ca-751b-0410-a591-d2e778427230
2006-08-01 03:36:36 +00:00
hieuhoang1972
33fa518c97 using mem pool. needs to be more finely tweaked to give good results
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@412 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-31 19:08:33 +00:00
hieuhoang1972
f403c2cbf6 getting ready to use mempool for Phrase
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@398 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-30 02:50:21 +00:00
hieuhoang1972
0a1e599120 Visual Studio issues
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@395 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-30 02:27:40 +00:00
hieuhoang1972
a8b4a4db4d Visual Studio issues
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@394 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-30 02:14:41 +00:00
hieuhoang1972
4192b9050f make mysql code work with new scxoring collection
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@384 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-29 16:28:56 +00:00
zens
dd124448c1 - bug fix and further cleanup
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@382 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-29 03:21:18 +00:00
redpony
ec776dd2b7 major score refactoring
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@371 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-29 01:11:45 +00:00
zens
dec0f5f703 print confusion net posterior in nbest list (if input is confusion net)
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@368 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-28 22:57:31 +00:00
redpony
ecabcbf0e0 add score logging
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@367 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-28 22:42:17 +00:00
hieuhoang1972
1df79b9d75 deleted Arc & LatticeEdge classes
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@364 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-28 22:11:20 +00:00
zens
0e6517b428 - fixing already fixed bugs that have been re-introduced by someone
- moving some stuff from PhraseDictionaryTree to GenerateTuples
 - removing some obsolete code


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@359 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-28 18:14:20 +00:00
hieuhoang1972
35f3c1b1b5 turned off profiling
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@349 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-28 03:18:18 +00:00
redpony
b48c33f1da use new score bookkeeping for nbest lists
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@346 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-28 01:15:09 +00:00
hieuhoang1972
cca42f4ba9 cvs ignore
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@340 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-27 20:39:31 +00:00
hieuhoang1972
08a2d7e14d cvs ignore
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@335 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-27 16:49:33 +00:00
hieuhoang1972
0c0ff800de Visual Studio issues
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@331 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-27 15:26:10 +00:00
redpony
14d342508e preparation for new ScoreComponentCollection, which will be much more efficient
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@329 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-27 13:47:43 +00:00
zens
4022ae95bc - print statistics for confusion nets
- small bug fixes


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@316 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-26 20:33:33 +00:00
hieuhoang1972
1ae613e79c perf profiling turned on in eclipse in debug conf
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@315 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-26 20:24:37 +00:00
redpony
b7b03f9ff2 rename function
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@302 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-26 14:29:45 +00:00
bojar
b0f7cb32cb Added -report-all-factors, so that stdout is more verbose.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@300 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-26 00:01:05 +00:00
phkoehn
f89ba84fdc reset sentence counter after reading phrases from input
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@293 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-25 21:53:50 +00:00
hieuhoang1972
c99e2c1786 Visual Studio issues
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@289 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-25 21:41:04 +00:00
bojar
88f4bced55 reimplemented future score matrix calculation, should be faster.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@288 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-25 21:17:12 +00:00
ccorbett
ee3e1e1576 Lexical reordering framework; confusion net still not working but Chris implied that was ok. All other regression tests passed
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@276 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-25 15:52:09 +00:00
redpony
c2be12bcd4 add irstlm support
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@273 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-25 14:42:49 +00:00
zens
a28dc42dd0 - fixed confusion net stuff (translations are identical, scores still different)
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@270 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-25 14:08:21 +00:00
zens
162c8583c6 - TranslationOptionCollection...
* moved generic functions to base class
   * todo: implement ProcessInitialTranslation and ProcessUnknownWord
           for confusion nets


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@269 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-25 13:35:42 +00:00
hieuhoang1972
dd8a983149 merged up of chaching algorithm. lot6s of changed made to manager.cpp, TranslationOptColl.cpp, amongst other.
other code may be broken as a result


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@267 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-25 04:53:29 +00:00
zens
287d01fab1 - weight for confusion net scores
- bug fix in ranking of target phrases for confusion nets
 - code cleanup


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@266 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-25 00:24:18 +00:00
bojar
2e83536ef1 Moses accepts -report-source-span 1 for a mose verbose output.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@265 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-24 23:48:18 +00:00
bojar
90124bd403 (useless) optimization flag added
+ reporting number od hypotheses


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@259 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-24 17:59:35 +00:00
zens
81a63989fa - do no read input phraes for confusion net input
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@257 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-21 22:19:02 +00:00
zens
47edc0599d - removed some obsolete parts
- avoided some code duplication
 - binary ttable is no longer a compiler switch:
   use binary ttable if exists, otherwise use standard ttable


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@253 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-21 22:03:03 +00:00
zens
98ef97e803 - translation of confusion nets integrated (works only with binary phrase table)
- removed some obsolete stuff


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@250 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-21 21:09:23 +00:00
hieuhoang1972
a857773715 don't output if not surface factor
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@248 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-21 17:57:37 +00:00
zens
70682b48a1 - prepare I/O for Confusion Nets
- remove some of the dynamic casting stuff


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@245 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-21 17:43:42 +00:00
zens
c220c86501 - binary phrase table now fully supported
-> define preprocessor variable USEBINTTABLE
      e.g. uncommenting the line in StaticData.cpp
           or on the make  command line
 - some clean up


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@241 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-21 13:50:41 +00:00
zens
4c03fc1de8 - removed processPhraseTable stuff
- explicit dependency on moses library
   -> automatically relink moses if libmoses.a changed


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@240 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-21 13:34:45 +00:00
zens
325353b0f4 - support for binary ttable in moses almost completed
- moved common part of CreateTranslationOptionCollection() for text and confusion net input
   to base class


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@237 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-21 13:22:00 +00:00
eherbst
7a6600617e minor cleaning-up
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@196 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-19 22:05:52 +00:00
zens
648bd1dfcd - prepared confusion net input
-> new base class InputType is used throughout the decoder
      instead of Sentence
      Sentence and ConfusionNet derive from this class
   -> Manager etc. do not know if the input is a sentence or a
      confusion net (but could check if REALLY needed)
 - two separate classes derived from TranslationOptionCollection:
   one for text input and one for confusion net input
 - score computation in PhraseDictionaryTree.cpp and some optimizations


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@183 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-19 00:16:29 +00:00
eherbst
7beabbd9ed adding source-word deletion (currently commented out) and unknown-word handling (also currently commented out)
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@175 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-18 18:11:09 +00:00
hieuhoang1972
b6e9b01155 checking eclipse project files
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@172 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-18 17:27:47 +00:00
alex_const
bcb49a09c6 fixed bug in hypothesis backtracking
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@151 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-17 15:05:00 +00:00
redpony
bf423cb0d3 temporary "fix" (break fewer people)
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@149 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-17 14:31:34 +00:00
hieuhoang1972
d655816734 for miscellaneous scripts and programs stuff
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@136 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-15 03:19:07 +00:00
zens
27dc2bfb7a *** empty log message ***
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@131 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-15 00:13:27 +00:00
zens
34b6f1d349 prefix tree phrase table:
- access to tree structure
  - tool for conversion
  - some code cleaning / reformatting


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@129 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-14 23:50:58 +00:00
hieuhoang1972
78e1a8830d commented out BackTrack
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@128 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-14 23:11:08 +00:00
hieuhoang1972
365460dff9 bug fixes & starting on translation option caching
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@125 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-14 22:21:54 +00:00
alex_const
2c0bfddfc2 Added hypothesis backtracking for the best hypothesis for verbose option 3.
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@123 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-14 21:51:05 +00:00
hieuhoang1972
afd1e85498 bug fixes & starting on translation option caching
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@115 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-14 18:13:50 +00:00
redpony
a08575e264 fix makefile stuff for moses-cmd
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@114 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-14 16:19:29 +00:00
eherbst
1bd381c35c removing various junk I and a couple others CVSed accidentally
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@108 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-14 15:17:59 +00:00
eherbst
1ec33be9d6 modularized unknown-source-word handling (see UnknownWordHandler)
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@107 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-14 14:36:51 +00:00
hieuhoang1972
edc5548cfe recommit config.in
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@103 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-14 14:01:52 +00:00
redpony
d7e5db0856 unify logging
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@102 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-14 13:31:06 +00:00
bojar
02a83cbc28 Never add a hypo with score worse than the worst in the (at least once pruned) stack, as it would get pruned anyway.
Renamed Add(hypo, beamthreshold) to AddPrune(hypo)


git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@97 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-13 21:23:42 +00:00
bojar
572f3a2dbe --enable-profiling option for ./configure
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@90 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-13 17:59:27 +00:00
redpony
d3f951ea85 add logging
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@86 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-13 12:09:23 +00:00
hieuhoang1972
e338a6dbc0 variable number of score components for phrase trans
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@79 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-12 10:16:21 +00:00
hieuhoang1972
038ef552b0 ScoreComponentCollection to be map !
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@77 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-12 09:44:11 +00:00
hieuhoang1972
38e4001a75 ScoreComponentCollection o be set instead of map
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@72 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-12 06:51:28 +00:00
hieuhoang1972
3e0e05ecde rolled back variable number of translation component scores
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@69 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-12 06:38:12 +00:00
hieuhoang1972
563094e3f8 visual studio proj
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@64 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-12 01:37:33 +00:00
hieuhoang1972
abd7a65b26 visual studio proj
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@63 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-12 01:30:26 +00:00
hieuhoang1972
90b5dadfe0 visual studio proj
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@62 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-12 00:59:55 +00:00
hieuhoang1972
007dca2a99 ScoreType namespace
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@45 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-11 00:22:21 +00:00
hieuhoang1972
9d4778441d Debugging
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@40 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-10 21:27:24 +00:00
redpony
37456ad0f4 allow static linking
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@38 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-10 20:52:20 +00:00
redpony
9278b06622 fixes for automake
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@35 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-10 19:54:11 +00:00
hieuhoang1972
79ec2cf2a8 Debugging
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@27 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-09 23:56:40 +00:00
hieuhoang1972
822963842b variable number of translation component scores
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@24 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-09 21:37:49 +00:00
hieuhoang1972
df0d858673 variable number of translation component scores
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@18 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-08 23:42:26 +00:00
hieuhoang1972
1b9437bfd5 variable number of translation component scores
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@15 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-08 04:44:12 +00:00
hieuhoang1972
af6d8fdb53 variable number of translation component scores
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@14 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-08 03:23:15 +00:00
hieuhoang1972
1138c0c0f7 variable number of translation component scores
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@8 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-07 16:46:30 +00:00
hieuhoang1972
dc1a0216ba put into sourceforge
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@5 1f5c12ca-751b-0410-a591-d2e778427230
2006-07-04 18:04:38 +00:00