google/sentencepiece
1
1
Fork 0
mirror of https://github.com/google/sentencepiece.git synced 2024-10-26 11:38:45 +03:00
Code Issues Packages Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
Taku Kudo 2018-06-09 00:32:55 +09:00 committed by GitHub
parent 53e4ae415e
commit a929b63458
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -268,7 +268,7 @@ The usage is basically the same as that of ```subword-nmt```. Assming that L1 an
% spm_encode --model=spm.model --generate_vocabulary < {train_file}.L2 > {vocab_file}.L2
```
```shuffle``` command is used just in case because ```spm_encode``` loads the first 10M lines of corpus by default.
```shuffle``` command is used just in case because ```spm_train``` loads the first 10M lines of corpus by default.
Then segment train/test corpus with ```--vocabulary``` option