BERT for packets: pre-training on encrypted traffic bursts

What it does

ET-BERT is a BERT-style transformer pre-trained on large-scale unlabelled encrypted traffic, then fine-tuned to classify specific traffic types. It learns contextual relationships between datagrams and transport bursts from raw packet features rather than decrypted payloads. The repository includes a downloadable pre-trained model and data pipelines built around a CSTNET TLS 1.3 dataset.

The interesting bit

The premise is that encrypted packet sequences carry enough structural signal—burst boundaries, timing, datagram order—to support masked pre-training tricks usually reserved for natural language. The authors call this “contextualized datagram representation,” which is academic speak for tokenizing packet captures and feeding them to a transformer.

Key highlights

• Pre-trained on an encrypted traffic burst corpus using multi-layer attention, then fine-tuned on small labeled datasets for specific scenarios.
• Ships with an anonymous real-world TLS 1.3 dataset (CSTNET) and a downloadable pre-trained checkpoint.
• Built on top of the UER-py framework, reusing its transformer and training machinery.
• Designed to be applied across different encrypted traffic scenarios without scenario-specific re-architecture.
• Uses WordPiece tokenization with a custom vocabulary (encryptd_vocab.txt) for traffic representation.

Caveats

• The codebase expects you to edit hardcoded paths inside source files before running anything.
• Full reproduction requires a specific hardware and software stack: Tesla V100S, CUDA 11.4, tshark, SplitCap, NVIDIA apex, and TensorFlow for model conversion.
• Only anonymous data is released from the original CSTNET dataset; using your own pcaps requires navigating the data_process scripts.

Verdict

A useful research artifact if you study encrypted traffic analysis or need a baseline for TLS 1.3 flow classification without decryption. For production use, the path-editing and dependency stack suggest waiting for a more packaged implementation.