Similar articles

Abstract: vision transformers (ViTs) have achieved remarkable success across various artificial intelligence-based computer vision applications. However, their demanding computational and memory requirements pose significant challenges for deployment on resource-constrained edge devices. Although post-training quantization (PTQ) provides a promising solution by reducing model precision with minimal calibration data, aggressive low-bit quantization typically leads to substantial performance degradation. To address this challenge, we present the truncated uniform-log2 quantizer and progressive bit-decline reconstruction method for vision transformers quantization (TP-ViT). It is an innovative PTQ framework specifically designed for ViTs, featuring two key technical contributions. 1) Truncated uniform-log2 quantizer: This novel quantization approach effectively handles outlier values in post-softmax activations, significantly reducing quantization errors. 2) Bit-decline optimization strategy: Our progressive quantization method employs transition weights to gradually reduce bit precision while maintaining model performance under extreme quantization conditions. Comprehensive experiments on image classification, object detection, and instance segmentation tasks demonstrate TP-ViTa??s superior performance compared to state-of-the-art PTQ methods, particularly in challenging 3-bit quantization scenarios. Our framework achieves a notable 6.18% improvement in Top-1 accuracy for ViT-small under 3-bit quantization. These results validate TP-ViTa??s robustness and general applicability, paving the way for more efficient deployment of ViTs models in computer vision applications on edge hardware.