Transformer-based English–Chinese Machine Translation & Attention Visualizer
A Transformer built from scratch — no nn.Transformer — with its attention
laid bare.
Overview
A from-scratch implementation of the Transformer architecture — scaled dot-product
attention, multi-head attention, positional encoding, and a full Encoder/Decoder, all
hand-built rather than assembled from PyTorch's built-in nn.Transformer.
It's trained on a large English–Chinese parallel corpus using custom byte-pair-encoding
(BPE) tokenizers for each language.
The Web Demo
- Translate any English sentence into Chinese, live
- Visualize the Encoder's self-attention weights as a heatmap
- See the BPE token breakdown alongside the heatmap
- A "simulated vs. real inference" comparison mode, built for teaching how the model actually attends to its input
Tech Stack
PyTorch, Flask, and custom BPE tokenizers for English and Chinese.