Brokentooth.io / 04 Models / Swipe Typing and the Keyboard Model

Swipe Typing and the Keyboard Model

Getting a gesture recognizer to work on glass takes more than a clever classifier.

A continuous swipe across an on-screen keyboard is not just a flurry of sloppy discrete taps. It is a continuous motor trajectory across physical space, which sounds elegant until you try to decode one on a phone screen without setting the battery on fire.

In OmniBoard, directional gestures have implemented code paths. Swiping across the spacebar moves the cursor; swiping left over backspace nukes a word. Those are simple vector thresholds pegged to unambiguous user actions. Continuous word-level glide typing, however, lives in an entirely different engineering reality. It remains operationally shelved, and for good reason.

Part of that reason is mundane geometric fragility. In the prototype pipeline, input coordinates live in a normalized space pegged to the alpha-key bounding box. But keyboard geometry on Android is rarely static: key heights shift, mod rows appear and disappear, and padding changes. If you normalize X and Y axes independently without aspect correction, an altered row height compresses the gesture into an anisotropic sliver where standard Euclidean distance thresholds stop meaning anything.

Then there is the modeling pipeline itself. Decoding a continuous curve into characters requires a two-stage contract. In research experiments on Titan, Stage 1 generates synthetic trajectories using minimum-jerk kinematics and the Two-Thirds Power Law to simulate human corner-cutting around key centroids. Stage 2 feeds those coordinate curves into NeuroSwipe, a compact 4-layer Transformer encoder-decoder trained on anonymous multi-session gesture corpora. The coordinate contract strips apostrophes from the trajectory geometry because nobody swipes through punctuation, but preserves them in the character vocabulary so a glide over dont emits don't.

To run that on mobile without runtime penalties, the model has to be split into two ONNX graphs: a static encoder evaluated once per gesture, and a single-step autoregressive decoder cell looped over candidate tokens. It works cleanly in an evaluation harness. But bridging the gap between promising offline loss curves and an everyday mobile keyboard that never stumbles over your thumb is an entirely different standard. Until the classifier matches that standard on metal, directional gestures stay live and continuous glide stays safely in the research branch.