mosesdecoder/vw/README.md

Vowpal Wabbit for Moses
=======================

This is an attempt to integrate Vowpal Wabbit with Moses as a stateless feature
function.

Compatible with this frozen version of VW:

    https://github.com/moses-smt/vowpal_wabbit

To enable VW, you need to provide a path where VW was installed (using `make install`) to bjam:

    ./bjam --with-vw=<path/to/vw/installation>

Implemented classifier features
-------------------------------

* `VWFeatureSourceBagOfWords`: This creates a feature of form bow^token for every
source sentence token.
* `VWFeatureSourceExternalFeatures column=0`: when used with -inputtype 5 (`TabbedSentence`) this can be used to supply additional feature to VW. The input is a tab-separated file, the first column is the usual input sentence, all other columns can be used for meta-data. Parameter column=0 counts beginning with the first column that is not the input sentence.
* `VWFeatureSourceIndicator`: Ass a feature for the whole source phrase.
* `VWFeatureSourcePhraseInternal`: Adds a separate feature for every word of the source phrase.
* `VWFeatureSourceWindow size=3`: Adds source words in a window of size 3 before and after the source phrase as features. These do not overlap with `VWFeatureSourcePhraseInternal`.
* `VWFeatureTargetIndicator`: Adds a feature for the whole target phrase.
* `VWFeatureTargetPhraseInternal`: Adds a separate feature for every word of the target phrase.

Configuration
-------------

To use the classifier edit your moses.ini

    [features]
    ...
    VW path=/home/username/vw/classifier1.vw
    VWFeatureSourceBagOfWords
    VWFeatureTargetIndicator
    VWFeatureSourceIndicator
    ...

    [weights]
    ...
    VW0= 0.2
    ...

If you change the name of the main VW feature, remember to tell the VW classifier
features which classifier they belong to:

    [features]
    ...
    VW name=bart path=/home/username/vw/classifier1.vw
    VWFeatureSourceBagOfWords used-by=bart
    VWFeatureTargetIndicator used-by=bart
    VWFeatureSourceIndicator used-by=bart
    ...

    [weights]
    ...
    bart= 0.2
    ...

You can also use multiple classifiers:

    [features]
    ...
    VW name=bart path=/home/username/vw/classifier1.vw
    VW path=/home/username/vw/classifier2.vw
    VW path=/home/username/vw/classifier3.vw
    VWFeatureSourceBagOfWords used-by=bart,VW0
    VWFeatureTargetIndicator used-by=VW1,VW0,bart
    VWFeatureSourceIndicator used-by=bart,VW1
    ...

    [weights]
    ...
    bart= 0.2
    VW0= 0.2
    VW1= 0.2
    ...

Features can use any combination of factors. Provide a comma-delimited list of factors in the `source-factors` or `target-factors` variables to override the default setting (`0`, i.e. the first factor).

Training the classifier
-----------------------

Training uses `vwtrainer` which is a limited version of the `moses` binary. To train, provide your training data as input in the following format:

    source tokens<tab>target tokens<tab>word alignment

Use Moses format for the word alignment (`0-0 1-0` etc.). Set the input type to 5 (`TabbedSentence`, see above):

    [inputtype]
    5

Configure your features in the `moses.ini` file (see above) and set the `train` flag:

     [features]
     ...
     VW name=bart path=/home/username/vw/features.txt train=1
     ...

The `path` variable points to the file (prefix) where features will be written. Currently, threads write to separate files (maybe subject to change sooner or later): `features.txt.1`, `features.txt.2` etc.

`vwtrainer` creates the translation option collection for each input sentence but does not run decoding. Therefore, you probably want to disable expensive feature functions such as the language model (LM score is not used by VW features at the moment).

Run `vwtrainer`:

    vwtrainer -f moses.trainvw.ini < tab-separated-training-data.tsv

Currently, classification is implemented using VW's `csoaa_ldf` scheme with quadratic features which take the product of the source namespace (`s`, contains label-independent features) and the target namespace (`t`,  contains label-dependent features).

To train VW in this setting, use the command:

    cat features.txt.* | vw --hash all --loss_function logistic --noconstant -b 26 -q st --csoaa_ldf mc -f classifier1.vw