mosesdecoder/scripts/share/nonbreaking_prefixes
Jeremy Gwinnup a5fb4d1550 Fixed bug in tokenizer.perl where comma separated lists of single
characters aren't handled correctly

input> A,B,C,D,E,F

yielded> A, B,C , D,E , F

now yields> A, B, C, D, E, F

Updated Russian nonbreaking prefixes list with capital letters
2013-08-16 14:39:50 -04:00
..
nonbreaking_prefix.ca lock m_vocab variable access in Encode() and Lookup(). Other functions are still not threadsafe 2012-06-26 13:33:34 -04:00
nonbreaking_prefix.cs czech prefixes 2012-07-14 03:32:10 +02:00
nonbreaking_prefix.de lock m_vocab variable access in Encode() and Lookup(). Other functions are still not threadsafe 2012-06-26 13:33:34 -04:00
nonbreaking_prefix.el lock m_vocab variable access in Encode() and Lookup(). Other functions are still not threadsafe 2012-06-26 13:33:34 -04:00
nonbreaking_prefix.en lock m_vocab variable access in Encode() and Lookup(). Other functions are still not threadsafe 2012-06-26 13:33:34 -04:00
nonbreaking_prefix.es lock m_vocab variable access in Encode() and Lookup(). Other functions are still not threadsafe 2012-06-26 13:33:34 -04:00
nonbreaking_prefix.fr lock m_vocab variable access in Encode() and Lookup(). Other functions are still not threadsafe 2012-06-26 13:33:34 -04:00
nonbreaking_prefix.hu Hungarian and Latvian non-breaking prefix files 2013-03-18 17:17:35 -04:00
nonbreaking_prefix.is lock m_vocab variable access in Encode() and Lookup(). Other functions are still not threadsafe 2012-06-26 13:33:34 -04:00
nonbreaking_prefix.it lock m_vocab variable access in Encode() and Lookup(). Other functions are still not threadsafe 2012-06-26 13:33:34 -04:00
nonbreaking_prefix.lv Hungarian and Latvian non-breaking prefix files 2013-03-18 17:17:35 -04:00
nonbreaking_prefix.nl lock m_vocab variable access in Encode() and Lookup(). Other functions are still not threadsafe 2012-06-26 13:33:34 -04:00
nonbreaking_prefix.pl Merge branch 'trunk' into miramerge 2012-07-17 13:36:50 +01:00
nonbreaking_prefix.pt lock m_vocab variable access in Encode() and Lookup(). Other functions are still not threadsafe 2012-06-26 13:33:34 -04:00
nonbreaking_prefix.ro lock m_vocab variable access in Encode() and Lookup(). Other functions are still not threadsafe 2012-06-26 13:33:34 -04:00
nonbreaking_prefix.ru Fixed bug in tokenizer.perl where comma separated lists of single 2013-08-16 14:39:50 -04:00
nonbreaking_prefix.sk lock m_vocab variable access in Encode() and Lookup(). Other functions are still not threadsafe 2012-06-26 13:33:34 -04:00
nonbreaking_prefix.sl lock m_vocab variable access in Encode() and Lookup(). Other functions are still not threadsafe 2012-06-26 13:33:34 -04:00
nonbreaking_prefix.sv lock m_vocab variable access in Encode() and Lookup(). Other functions are still not threadsafe 2012-06-26 13:33:34 -04:00
README.txt lock m_vocab variable access in Encode() and Lookup(). Other functions are still not threadsafe 2012-06-26 13:33:34 -04:00

The language suffix can be found here:

http://www.loc.gov/standards/iso639-2/php/code_list.php