12989 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
XapaJIaMnu	0451142ece	Add null token normalization for models to be used with the chart decoder.	2014-11-13 16:13:38 +00:00
XapaJIaMnu	aae894fe6b	Add null token in vocabulary during construction	2014-11-13 16:13:38 +00:00
XapaJIaMnu	b4f51c05d1	Add option to reduce the ngrams from already prepared .ngrams file to train a model with smaller number of ngrams	2014-11-13 16:13:38 +00:00
XapaJIaMnu	5851a2c2bb	Prevent realocation of vectors	2014-11-13 16:13:38 +00:00
Paul Baltescu	2705a47876	Fix OxLM.	2014-11-13 16:13:38 +00:00
XapaJIaMnu	02c375ef78	Refactor the BilingualLM for chart to make it faster. Untested	2014-11-13 16:13:04 +00:00
XapaJIaMnu	7858f74e9e	Rename BilingualLM_NPLM so that it is not confused with a sparse feature	2014-11-13 16:13:04 +00:00
Paul Baltescu	61826cee8a	Rename OxLM features.	2014-11-13 16:13:04 +00:00
Paul Baltescu	32c169c25f	Optional back-off to POS tags in OxLM.	2014-11-13 16:12:19 +00:00
Paul Baltescu	86d64b65e2	Correctly map the source unknown token.	2014-11-13 16:10:40 +00:00
Paul Baltescu	167e272818	Fix Bilingual OxLM context word order.	2014-11-13 16:10:40 +00:00
Paul Baltescu	5f9d481ee6	Make query cache sentence specific.	2014-11-13 16:10:40 +00:00
Paul Baltescu	248aa4bf8a	Convert moses words to oxlm word ids.	2014-11-13 16:10:40 +00:00
Paul Baltescu	97b632e045	Clean up OxLMMapper.	2014-11-13 16:08:56 +00:00
Paul Baltescu	90ebf13789	Set BilingualLM parameters nicely.	2014-11-13 16:08:10 +00:00
Paul Baltescu	7588c4b8e3	Skeleton for source conditioned OxLM feature.	2014-11-13 16:08:10 +00:00
Paul Baltescu	6f9d59129f	Rename LBLLM -> OxLM.	2014-11-13 16:07:38 +00:00
Paul Baltescu	af28063e3b	Fix compilation errors introduced by new oxlm changes.	2014-11-13 15:54:54 +00:00
Paul Baltescu	4811701277	Fix broken include.	2014-11-13 15:51:48 +00:00
Paul Baltescu	cb7167f088	Fix bugs in BilingualLM for chart based decoding.	2014-11-13 15:51:48 +00:00
Paul Baltescu	3624bd776c	Fix a few bugs in BilingualLM for phrase based decoding.	2014-11-13 15:51:48 +00:00
Paul Baltescu	5f87cf94d8	Move BilingualLM under LM.	2014-11-13 15:51:48 +00:00
XapaJIaMnu	fbac0ae418	Make sure we always have unk in the vocabulary, otherwise we get off-by-one indexes during decoding	2014-11-13 15:51:48 +00:00
XapaJIaMnu	961578286f	Forgot to close a file...	2014-11-13 15:51:48 +00:00
XapaJIaMnu	1bac666e5f	Fix small oversights	2014-11-13 15:51:48 +00:00
XapaJIaMnu	617ef015df	Extend train_nplm with various options	2014-11-13 15:51:48 +00:00
XapaJIaMnu	a1a10a9209	Remove unused variable that likely causes crashes	2014-11-13 15:51:48 +00:00
Nikolay Bogoychev	2b2766cce8	For GPU training one thread is optimal	2014-11-13 15:51:48 +00:00
Abmayne	4af68a0d1a	Barry's training scripts with some minor changes by me	2014-11-13 15:51:48 +00:00
XapaJIaMnu	97b7c766e4	fix vectors not being cleared	2014-11-13 15:51:47 +00:00
XapaJIaMnu	5d1584a3d8	Make it possible to have backend specific parameters.	2014-11-13 15:51:47 +00:00
XapaJIaMnu	c2f0419089	Fix compilation	2014-11-13 15:51:47 +00:00
XapaJIaMnu	0271b451b7	Some file renaming and fixes	2014-11-13 15:51:47 +00:00
XapaJIaMnu	af8ee9da43	Abstract BilingualLM. Working	2014-11-13 15:51:47 +00:00
XapaJIaMnu	bceff44fcb	Abstract BilingualLM to Binlm_NPLM. Doesn't compile	2014-11-13 15:51:47 +00:00
XapaJIaMnu	46d8987635	Start abstracting NPLM from BilingualLM	2014-11-13 15:51:47 +00:00
XapaJIaMnu	e449df548a	Fix a bunch of off by one and silly errors	2014-11-13 15:51:47 +00:00
XapaJIaMnu	38197698e0	Fix silly mistake in getTargetPhraseChart	2014-11-13 15:51:47 +00:00
XapaJIaMnu	ea03722aaa	Fix a stupid mistake in TargetWords	2014-11-13 15:51:47 +00:00
XapaJIaMnu	7e26228fd7	Add support for moses_chart. Cleaning up needed.	2014-11-13 15:51:47 +00:00
Hieu Hoang	f76e1e8a03	compiles on gcc 4.6.3	2014-11-13 15:51:47 +00:00
XapaJIaMnu	62ec653df8	More work, source_ngrams missing	2014-11-13 15:51:47 +00:00
XapaJIaMnu	2979a617a9	int -> size_t	2014-11-13 15:51:47 +00:00
XapaJIaMnu	6e148defba	Rename variable to better represent what it means. Start syntactic decoding	2014-11-13 15:51:47 +00:00
XapaJIaMnu	ea01ede796	Support for POS backoff in factored models. Spaghetti code needs refactoring	2014-11-13 15:51:46 +00:00
XapaJIaMnu	caf797a6a4	GetNeuralLMId requries word instead of factor	2014-11-13 15:51:46 +00:00
XapaJIaMnu	b5e5e12c9f	Make some parameters of neuralLM optional	2014-11-13 15:51:46 +00:00
XapaJIaMnu	59ab036eb5	Get state, correct	2014-11-13 15:51:46 +00:00
XapaJIaMnu	4309903d6e	State, almost correct or correct	2014-11-13 15:51:46 +00:00
XapaJIaMnu	c225fb27d1	Fix small indexing error	2014-11-13 15:51:46 +00:00