Barry Haddow 34b139e2ae Remove debug		2015-02-13 12:14:18 +00:00
..
basic-protected-patterns	makemteval and small change to tokenizer. /Tom Hoar and Tomas Fulajtar	2014-11-21 13:55:13 +00:00
deescape-special-chars-PTB.perl	Penn Tree Bank compliant versions of preprocessing	2014-10-10 16:49:06 +01:00
deescape-special-chars.perl	escape bar character with proper html escape sequence	2012-06-25 23:37:59 +01:00
detokenizer.perl	Add option to do Penn Treebank style tokenization	2013-07-24 13:41:21 +01:00
escape-special-chars.perl	bug fix	2012-06-26 22:49:59 +01:00
lowercase.perl	add lowercaser	2010-08-02 14:05:23 +00:00
normalize-punctuation.perl	Remove debug	2015-02-13 12:14:18 +00:00
pre-tokenizer.perl	Added -b switch to pretokenizer to allow disabling of buffering.	2014-04-16 03:28:16 +01:00
remove-non-printing-char.perl	script to remove non-printing characters once and for all	2014-05-31 00:24:15 +01:00
replace-unicode-punctuation.perl	added replace-unicode-punctuation.perl	2013-11-04 21:46:36 +00:00
tokenizer_PTB.perl	move normalisation of quotes into normalize-punctuation.perl /Tom Hoar	2015-01-16 11:37:31 +00:00
tokenizer.perl	"just put it in. I'll verify it if i can be bovvered" --Hieu /usr/bin/env	2015-01-29 18:37:05 -05:00