113 Commits

Author	SHA1	Message	Date
phikoehn	b275c94dbf	allow for inclusion of extract from previous run	2012-12-12 07:02:59 +00:00
Hieu Hoang	487822ed14	don't write to stdout	2012-11-22 15:08:00 +00:00
phikoehn	5cd614ecd8	adjust to irstlm changes	2012-11-20 17:18:57 +00:00
Hieu Hoang	f3ec76ac56	minor change to calling irst training	2012-10-14 19:51:46 +01:00
phikoehn	04544f8bfc	better error message when reference file not found	2012-10-04 23:22:19 +01:00
Lane Schwartz	7b042edc6c	Send stderr to /dev/null when looking for pawd. ... This cleans up the logs a bit for those of us who don't have pawd. Otherwise, messages like the following show up in the logs: /usr/bin/which: no pawd in ...	2012-09-28 14:55:09 -04:00
Lane Schwartz	a323c8daf7	Send stderr to /dev/null when looking for pawd. ... This cleans up the logs a bit for those of us who don't have pawd. Otherwise, messages like the following show up in the logs: /usr/bin/which: no pawd in ... bash: pawd: command not found	2012-09-28 14:37:53 -04:00
phikoehn	a84fbcb80a	bug fix for using domain feature in multi-process extract	2012-09-24 11:14:36 +01:00
Hieu Hoang	30e5b0575b	merge conflict	2012-09-03 19:12:00 +01:00
phikoehn	4a1a995878	a lot of changes	2012-08-18 23:48:26 +01:00
phikoehn	366ab93f8a	a lot of changes	2012-08-18 23:47:05 +01:00
Hieu Hoang	a8117c0afd	redirect more to stderr. Split input extract files every 50m lines, instead of 10m lines	2012-08-08 14:15:31 +01:00
Hieu Hoang	efdb983e0c	redirect parallel scoring to standard err, appending to existing	2012-08-07 15:38:42 +01:00
Hieu Hoang	a1ab8e354a	cleanup of variables. Need to delete temporary files	2012-07-31 02:21:48 +01:00
Hieu Hoang	23bbf12752	redirect stderr from extract program to stderr of script that runs it, for easier debugging	2012-07-18 22:53:47 +01:00
Hieu Hoang	f0a3e67717	redirect stderr from extract program to stderr of script that runs it, for easier debugging. Recursive create of output directory, in case output directory doesn't exist	2012-07-18 22:33:49 +01:00
Hieu Hoang	b609473645	under changes to make scripts handle directories with spaces. Karel gave up, and it messing up cruise control	2012-07-18 11:02:24 +01:00
Karel Bílek	bc64284b31	Merge remote-tracking branch 'upstream/master'	2012-07-17 04:37:16 +02:00
Karel Bílek	ad257b27d9	spaces in filepath does not wreak havoc anymore	2012-07-17 04:36:27 +02:00
Hieu Hoang	8180181a89	Merge https://github.com/moses-smt/mosesdecoder	2012-07-10 09:19:31 +01:00
Hieu Hoang	7d982bcd8e	comments	2012-07-10 09:18:37 +01:00
phikoehn	51589a25e7	fix for leading, trailing, and multiple spaces, suggested by Yang Gao	2012-07-06 05:12:50 +01:00
Hieu Hoang	debe090426	Change Bin to RealBin. Thanks to Tom Hoar	2012-06-26 11:57:23 -04:00
phikoehn	7cd8356752	minor bug fixes	2012-06-25 23:38:46 +01:00
Barry Haddow	25f5def387	Fix for case where no ttable is to be created	2012-06-03 22:03:41 +01:00
phikoehn	6d1165654c	script updates and added ems config help	2012-05-28 20:15:58 +01:00
Hieu Hoang	4c90c88733	make phrase-table.half naming consistent. Requested by Phil Williams	2012-05-26 12:21:32 +01:00
Hieu Hoang	e42f8d36ae	parallel scoring	2012-05-24 15:20:23 +01:00
Hieu Hoang	d979b4454d	parallel scoring	2012-05-24 15:05:23 +01:00
Hieu Hoang	f91b19fa41	Works with phrase-table scoring	2012-05-23 19:26:37 +01:00
Hieu Hoang	761ac4f0fc	parallel extract. Works with reordering	2012-05-23 19:02:36 +01:00
Hieu Hoang	776dd7856e	add script to reverse word alignment	2012-05-17 17:59:48 +01:00
Hieu Hoang	e74f67abc5	use new gzipped extract file. Always save extract files already sorted	2012-05-13 10:23:59 +01:00
Hieu Hoang	066c209fcb	use new gzipped extract file. Always save extract files already sorted	2012-05-12 15:41:07 +01:00
Hieu Hoang	2bcaa00dbd	more comment for irst training wrapper	2012-05-04 11:09:12 +01:00
Hieu Hoang	ef390a705f	1. use new gsort, rather than sort, if it is available ... 2. check return value of cat command in extract-parallel in case it runs out of disk space etc	2012-03-11 15:50:46 +07:00
Hieu Hoang	de9eeab7e9	mac osx compatible split & sort	2012-02-23 13:26:19 +00:00
Hieu Hoang	7073f7d891	bug fix by Guchun Zhang	2012-02-16 10:28:09 +07:00
Hieu Hoang	7f6f8a99f9	bug fix by Guchun Zhang	2012-02-16 10:26:48 +07:00
Hieu Hoang	53b41f7c45	parallel extract	2012-02-09 18:24:49 +07:00
root	4e93cda304	make irst training a little more compatible with sri	2012-02-06 16:53:41 +07:00
Hieu Hoang	3de86c5724	undo start-translation-id arg, as per email from Sandra Noubours	2012-01-26 21:24:05 +07:00
Hieu Hoang	89d4743bad	rollback accidental commit	2012-01-24 18:04:44 +07:00
Hieu Hoang	5ae2a24878	add --sort-batch-size arg to train-model.perl	2012-01-24 18:00:45 +07:00
Philipp Koehn	cdf735b01b	better error message when no corpus defined, better integration of IRSTLM training	2011-12-21 05:50:59 +00:00
Philipp Koehn	b95c372e3a	updates to EMS: mteval-v13a.pl, parallel preparation, better paths and defaults in examples	2011-12-21 04:26:27 +00:00
bhaddow	b0e0546dd9	Add option to lowercase input	2011-12-16 17:00:13 +00:00
Hieu Hoang	21009b5d1e	revert	2011-12-12 20:48:42 +07:00
root	9b58880a32	script to train LM with IRST in the EMS. A bit of a hack to make it look like SRI	2011-12-12 13:32:20 +07:00
Hieu Hoang	712bbf199c	load hiero format for reading suffix arrays	2011-11-14 00:27:18 +07:00