The Encoder model is a universal layout-agnostic and language-agnostic, and is used for making swipe typing predictions in the general case. However, it does not offer cutting-edge accuracy.

The ContextLM model is a very small language model that is trained for a single language. It's used to improve the quality of predictions by eliminating nonsensical words given the preceding words in the sentence. It only requires text data for training.

Finally, the decoder is a language-specific and layout-specific model that learns layout's peculiarities and achieves leading accuracy. As it requires swipe typing data for a specific layout and language for training, we only have a QWERTY English decoder for now.

With all 3 models and with a beam width of 300, we achieve a top-4 fail rate of only ~4% on our test set. Ignoring out-of-vocabulary cases, the error rate is below 1%.