moses-smt/mosesdecoder
1
1
Fork 0
mirror of https://github.com/moses-smt/mosesdecoder.git synced 2024-08-16 23:10:31 +03:00
Code Issues Packages Projects Releases Wiki Activity

Modernized

Browse Source 
I wanted to properly parse links on https://dumps.wikimedia.org/mirrors.html when page copied as text
My proposed changes does the job.
Basically I had to change by replacing the + at end of line 5 with *(\/)?
The pipe symbol could lead to crashes why I broke up line 5 to three lines. I suggest not using the pipe (|) after reading various posts.
This commit is contained in:
HjalmarrSv 2019-12-17 20:40:51 +01:00 committed by GitHub
parent a89691fee3
commit fa747062dc
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
scripts/tokenizer/basic-protected-patterns
View File

@ -2,4 +2,6 @@
<\S+( [a-zA-Z0-9]+\=\"?[^\"]\")+ ?\/?>
<\S+( [a-zA-Z0-9]+\=\'?[^\']\')+ ?\/?>
[\w\-\_\.]+\@([\w\-\_]+\.)+[a-zA-Z]{2,}
(http[s]?|ftp):\/\/[^:\/\s]+(\/\w+)*\/[\w\-\.]+
http[s]?:\/\/[^:\/\s]+(\/\w+)*\/[\w\-\.]*(\/)?
ftp[s]?:\/\/[^:\/\s]+(\/\w+)*\/[\w\-\.]*(\/)?
rsync:\/\/[^:\/\s]+(\/\w+)*\/[\w\-\.]*(\/)?