8 Commits

Author	SHA1	Message	Date
Jeroen Vermeulen	32722ab5b1	Support tokenize(const std::string &) as well. ... Convenience wrapper: the actual function takes a const char[], but many of the call sites want to pass a string and have to call its c_str() first.	2015-04-22 10:35:18 +07:00
Jeroen Vermeulen	b2d821a141	Unify tokenize() into util, and unit-test it. ... The duplicate definition works fine in environments where the inline definition becomes a weak symbol in the object file, but if it gets generated as a regular definition, the duplicate definition causes link problems. In most call sites the return value could easily be made const, which gives both the reader and the compiler a bit more certainty about the code's intentions. In theory this may help performance, but it's mainly for clarity. The comments are based on reverse-engineering, and the unit tests are based on the comments. It's possible that some of what's in there is not essential, in which case, don't feel bad about changing it! I left a third identical definition in place, though I updated it with my changes to avoid creeping divergence, and noted the duplication in a comment. It would be nice to get rid of this definition as well, but it'd introduce headers from the main Moses tree into biconcor, which may be against policy.	2015-04-22 09:59:05 +07:00
Hieu Hoang	05ead45e71	beautify	2015-01-14 11:07:42 +00:00
Nicola Bertoldi	e4eb201c52	merged master into dynamic-models and solved conflicts	2014-12-13 12:52:47 +01:00
Hieu Hoang	1b667e3e24	delete any mention of SAFE_GETLINE so it doesn't reappear	2014-06-08 17:07:12 +01:00
Hieu Hoang	cb94a3181b	use standard c++ getline instead of old Moses SAFE_GETLINE	2014-06-08 16:23:14 +01:00
Nicola Bertoldi	20b3e8929e	beautify	2014-05-19 15:35:08 +02:00
Matthias Huck	86ee3e15a4	new version of the score tool ... which is now capable of dealing with additional properties in an appropriate manner	2014-01-29 18:37:42 +00:00