Vision Transformers(ViTs)have achieved remarkable success across various artificial intelligence-based computer vision applications.However,their demanding computational and memory requirements pose significant challe...Vision Transformers(ViTs)have achieved remarkable success across various artificial intelligence-based computer vision applications.However,their demanding computational and memory requirements pose significant challenges for de-ployment on resource-constrained edge devices.Although post-training quantization(PTQ)provides a promising solution by reducing model precision with minimal calibration data,aggressive low-bit quantization typically leads to substantial perfor-mance degradation.To address this challenge,we present the truncated uniform-log2 quantizer and progressive bit-decline reconstruction method for vision Transformer quantization(TP-ViT).It is an innovative PTQ framework specifically designed for ViTs,featuring two key technical contributions:(1)truncated uniform-log2 quantizer,a novel quantization approach which effectively handles outlier values in post-Softmax activations,significantly reducing quantization errors;(2)bit-decline optimiza-tion strategy,which employs transition weights to gradually reduce bit precision while maintaining model performance under extreme quantization conditions.Comprehensive experiments on image classification,object detection,and instance segmenta-tion tasks demonstrate TP-ViT’s superior performance compared to state-of-the-art PTQ methods,particularly in challenging 3-bit quantization scenarios.Our framework achieves a notable 6.18 percentage points improvement in top-1 accuracy for ViT-small under 3-bit quantization.These results validate TP-ViT’s robustness and general applicability,paving the way for more efficient deployment of ViT models in computer vision applications on edge hardware.展开更多
Quantization noise caused by analog-to-digital converter(ADC)gives rise to the reliability performance degradation of communication systems.In this paper,a quantized non-Hermitian symmetry(NHS)orthogonal frequency-div...Quantization noise caused by analog-to-digital converter(ADC)gives rise to the reliability performance degradation of communication systems.In this paper,a quantized non-Hermitian symmetry(NHS)orthogonal frequency-division multiplexing-based visible light communication(OFDM-VLC)system is presented.In order to analyze the effect of the resolution of ADC on NHS OFDM-VLC,a quantized mathematical model of NHS OFDM-VLC is established.Based on the proposed quantized model,a closed-form bit error rate(BER)expression is derived.The theoretical analysis and simulation results both confirm the effectiveness of the obtained BER formula in high-resolution ADC.In addition,channel coding is helpful in compensating for the BER performance loss due to the utilization of lower resolution ADC.展开更多
The rapid growth of digital data necessitates advanced natural language processing(NLP)models like BERT(Bidi-rectional Encoder Representations from Transformers),known for its superior performance in text classificati...The rapid growth of digital data necessitates advanced natural language processing(NLP)models like BERT(Bidi-rectional Encoder Representations from Transformers),known for its superior performance in text classification.However,BERT’s size and computational demands limit its practicality,especially in resource-constrained settings.This research compresses the BERT base model for Bengali emotion classification through knowledge distillation(KD),pruning,and quantization techniques.Despite Bengali being the sixth most spoken language globally,NLP research in this area is limited.Our approach addresses this gap by creating an efficient BERT-based model for Bengali text.We have explored 20 combinations for KD,quantization,and pruning,resulting in improved speedup,fewer parameters,and reduced memory size.Our best results demonstrate significant improvements in both speed and efficiency.For instance,in the case of mBERT,we achieved a 3.87×speedup and 4×compression ratio with a combination of Distil+Prune+Quant that reduced parameters from 178 to 46 M,while the memory size decreased from 711 to 178 MB.These results offer scalable solutions for NLP tasks in various languages and advance the field of model compression,making these models suitable for real-world applications in resource-limited environments.展开更多
This paper proposes a novel method for the automatic diagnosis of keratitis using feature vector quantization and self-attention mechanisms(ADK_FVQSAM).First,high-level features are extracted using the DenseNet121 bac...This paper proposes a novel method for the automatic diagnosis of keratitis using feature vector quantization and self-attention mechanisms(ADK_FVQSAM).First,high-level features are extracted using the DenseNet121 backbone network,followed by adaptive average pooling to scale the features to a fixed length.Subsequently,product quantization with residuals(PQR)is applied to convert continuous feature vectors into discrete features representations,preserving essential information insensitive to image quality variations.The quantized and original features are concatenated and fed into a self-attention mechanism to capture keratitis-related features.Finally,these enhanced features are classified through a fully connected layer.Experiments on clinical low-quality(LQ)images show that ADK_FVQSAM achieves accuracies of 87.7%,81.9%,and 89.3% for keratitis,other corneal abnormalities,and normal corneas,respectively.Compared to DenseNet121,Swin transformer,and InceptionResNet,ADK_FVQSAM improves average accuracy by 3.1%,11.3%,and 15.3%,respectively.These results demonstrate that ADK_FVQSAM significantly enhances the recognition performance of keratitis based on LQ slit-lamp images,offering a practical approach for clinical application.展开更多
The Internet of Things(IoT)technology provides data acquisition,transmission,and analysis to control rehabilitation robots,encompassing sensor data from the robots as well as lidar signals for trajectory planning(desi...The Internet of Things(IoT)technology provides data acquisition,transmission,and analysis to control rehabilitation robots,encompassing sensor data from the robots as well as lidar signals for trajectory planning(desired trajectory).In IoT rehabilitation robot systems,managing nonvanishing uncertainties and input quantization is crucial for precise and reliable control performance.These challenges can cause instability and reduced effectiveness,particularly in adaptive networked control.This paper investigates networked control with guaranteed performance for IoT rehabilitation robots under nonvanishing uncertainties and input quantization.First,input quantization is managed via a quantization-aware control design,ensur stability and minimizing tracking errors,even with discrete control inputs,to avoid chattering.Second,the method handles nonvanishing uncertainties by adjusting control parameters via real-time neural network adaptation,maintaining consistent performance despite persistent disturbances.Third,the control scheme guarantees the desired tracking performance within a specified time,with all signals in the closed-loop system remaining uniformly bounded,offering a robust,reliable solution for IoT rehabilitation robot control.The simulation verifies the benefits and efficacy of the proposed control strategy.展开更多
Formal state space models of quantum control systems are deduced and a scheme to establish formal state space models via quantization could been obtained for quantum control systems is proposed. State evolution of qua...Formal state space models of quantum control systems are deduced and a scheme to establish formal state space models via quantization could been obtained for quantum control systems is proposed. State evolution of quantum control systems must accord with Schrdinger equations, so it is foremost to obtain Hamiltonian operators of systems. There are corresponding relations between operators of quantum systems and corresponding physical quantities of classical systems, such as momentum, energy and Hamiltonian, so Schrdinger equation models of corresponding quantum control systems via quantization could been obtained from classical control systems, and then establish formal state space models through the suitable transformation from Schrdinger equations for these quantum control systems. This method provides a new kind of path for modeling in quantum control.展开更多
A fast encoding algorithm based on the mean square error （MSE） distortion for vector quantization is introduced. The vector, which is effectively constructed with wavelet transform （WT） coefficients of images, can...A fast encoding algorithm based on the mean square error （MSE） distortion for vector quantization is introduced. The vector, which is effectively constructed with wavelet transform （WT） coefficients of images, can simplify the realization of the non-linear interpolated vector quantization （NLIVQ） technique and make the partial distance search （PDS） algorithm more efficient. Utilizing the relationship of vector L2-norm and its Euclidean distance, some conditions of eliminating unnecessary codewords are obtained. Further, using inequality constructed by the subvector L2-norm, more unnecessary codewords are eliminated. During the search process for code, mostly unlikely codewords can be rejected by the proposed algorithm combined with the non-linear interpolated vector quantization technique and the partial distance search technique. The experimental results show that the reduction of computation is outstanding in the encoding time and complexity against the full search method.展开更多
Information hiding schemes based on vector quantization （VQ） usually require lengthy VQ encoding and decoding processes. In this paper, we propose an efficient information hiding method based on closest paired tree ...Information hiding schemes based on vector quantization （VQ） usually require lengthy VQ encoding and decoding processes. In this paper, we propose an efficient information hiding method based on closest paired tree structure vector quantization （CPTSVQ）. The simulation result shows that the execution time of the proposed scheme is much shorter than that attained by previous approaches.展开更多
合法通信双方可以利用无线信道状态的随机特性提取符合信息论安全的共享密钥序列。为了提高无线信道提取密钥的效率,提出一种基于差分的多级量化共享密钥提取方案。该方案采用随机调制对无线信道进行高频采样,并引入融合随机抽样差分的...合法通信双方可以利用无线信道状态的随机特性提取符合信息论安全的共享密钥序列。为了提高无线信道提取密钥的效率,提出一种基于差分的多级量化共享密钥提取方案。该方案采用随机调制对无线信道进行高频采样,并引入融合随机抽样差分的自适应符号量化(ASQ)和均衡化多比特修正量化(BMMQ)这2个算法处理一阶差分序列,以获得原始密钥序列。在此基础上,应用信息协商算法纠正原始密钥中不一致的比特,并使用原始密钥及一阶差分序列重构信号,再对该信号进行二次量化,最终实现合法通信双方的密钥同步。实验结果表明,随机抽样差分能够将相邻样本点之间的相关系数降低至e^(-1)以下,有效降低密钥序列中的统计依赖性;在信噪比(SNR)为25 dB的条件下,ASQ算法可在保持原始密钥提取率(OKER)为0.86的同时,将密钥不一致率(KDR)降低至3.8×10^(-5);在无损量化的条件下,BMMQ算法可以把KDR降低至7×10^(-3)。最终生成的共享密钥序列通过了NIST(National Institute of Standards and Technology)随机性测试,验证了密钥的安全性和有效性。展开更多
随着大语言模型(large language model,LLM)参数规模的指数级增长,模型部署和推理面临着严峻的内存和计算资源挑战。量化技术作为模型压缩的核心方法,通过降低权重和激活值的数值精度,显著减少了模型的存储需求和计算开销。首先回顾了...随着大语言模型(large language model,LLM)参数规模的指数级增长,模型部署和推理面临着严峻的内存和计算资源挑战。量化技术作为模型压缩的核心方法,通过降低权重和激活值的数值精度,显著减少了模型的存储需求和计算开销。首先回顾了量化技术的发展历程,从经典的Int8/4量化方法到前沿的超低比特量化算法,总结了典型方法的技术特征与性能演进规律,指出传统实数域量化在极低比特条件下存在受限于离散化误差的挑战,难以突破性能上限。为此,进而系统性地梳理了复域量化系列工作。该系列工作提出了基于复数域的量化范式,通过在参数表示中引入幅度与相位2个自由度,显著扩展了模型的表达空间;此外,类比信号处理中通过将时域信号进行傅里叶变换与低通滤波实现稳定表示的经典范式,进一步提出了由实数模型经复域变换与复域量化,达成了无乘法稳定推理的技术路线。实验结果表明,该方案在多个基准数据集上优于现有超低比特量化方法,有效突破了实数域模型的性能天花板,展现出复域量化在高效建模与性能保持方面的潜在价值。总体而言,通过对量化技术演进及复域量化系列研究的系统分析,旨在揭示超低比特量化的发展规律与未来趋势,为高效大模型的理论研究与工程实现提供参考。展开更多
基金supported by the National Natural Science Foundation of China(Nos.62301092 and 62301093).
文摘Vision Transformers(ViTs)have achieved remarkable success across various artificial intelligence-based computer vision applications.However,their demanding computational and memory requirements pose significant challenges for de-ployment on resource-constrained edge devices.Although post-training quantization(PTQ)provides a promising solution by reducing model precision with minimal calibration data,aggressive low-bit quantization typically leads to substantial perfor-mance degradation.To address this challenge,we present the truncated uniform-log2 quantizer and progressive bit-decline reconstruction method for vision Transformer quantization(TP-ViT).It is an innovative PTQ framework specifically designed for ViTs,featuring two key technical contributions:(1)truncated uniform-log2 quantizer,a novel quantization approach which effectively handles outlier values in post-Softmax activations,significantly reducing quantization errors;(2)bit-decline optimiza-tion strategy,which employs transition weights to gradually reduce bit precision while maintaining model performance under extreme quantization conditions.Comprehensive experiments on image classification,object detection,and instance segmenta-tion tasks demonstrate TP-ViT’s superior performance compared to state-of-the-art PTQ methods,particularly in challenging 3-bit quantization scenarios.Our framework achieves a notable 6.18 percentage points improvement in top-1 accuracy for ViT-small under 3-bit quantization.These results validate TP-ViT’s robustness and general applicability,paving the way for more efficient deployment of ViT models in computer vision applications on edge hardware.
基金supported by the National Natural Science Foundation of China(No.62201508)the Zhejiang Provincial Natural Science Foundation of China(Nos.LZ21F010001 and LQ23F010004)the State Key Laboratory of Millimeter Waves of Southeast University,China(No.K202212).
文摘Quantization noise caused by analog-to-digital converter(ADC)gives rise to the reliability performance degradation of communication systems.In this paper,a quantized non-Hermitian symmetry(NHS)orthogonal frequency-division multiplexing-based visible light communication(OFDM-VLC)system is presented.In order to analyze the effect of the resolution of ADC on NHS OFDM-VLC,a quantized mathematical model of NHS OFDM-VLC is established.Based on the proposed quantized model,a closed-form bit error rate(BER)expression is derived.The theoretical analysis and simulation results both confirm the effectiveness of the obtained BER formula in high-resolution ADC.In addition,channel coding is helpful in compensating for the BER performance loss due to the utilization of lower resolution ADC.
文摘The rapid growth of digital data necessitates advanced natural language processing(NLP)models like BERT(Bidi-rectional Encoder Representations from Transformers),known for its superior performance in text classification.However,BERT’s size and computational demands limit its practicality,especially in resource-constrained settings.This research compresses the BERT base model for Bengali emotion classification through knowledge distillation(KD),pruning,and quantization techniques.Despite Bengali being the sixth most spoken language globally,NLP research in this area is limited.Our approach addresses this gap by creating an efficient BERT-based model for Bengali text.We have explored 20 combinations for KD,quantization,and pruning,resulting in improved speedup,fewer parameters,and reduced memory size.Our best results demonstrate significant improvements in both speed and efficiency.For instance,in the case of mBERT,we achieved a 3.87×speedup and 4×compression ratio with a combination of Distil+Prune+Quant that reduced parameters from 178 to 46 M,while the memory size decreased from 711 to 178 MB.These results offer scalable solutions for NLP tasks in various languages and advance the field of model compression,making these models suitable for real-world applications in resource-limited environments.
基金supported by the National Natural Science Foundation of China(Nos.62276210,82201148 and 62376215)the Key Research and Development Project of Shaanxi Province(No.2025CY-YBXM-044)+3 种基金the Natural Science Foundation of Zhejiang Province(No.LQ22H120002)the Medical Health Science and Technology Project of Zhejiang Province(Nos.2022RC069 and 2023KY1140)the Natural Science Foundation of Ningbo(No.2023J390)the Ningbo Top Medical and Health Research Program(No.2023030716).
文摘This paper proposes a novel method for the automatic diagnosis of keratitis using feature vector quantization and self-attention mechanisms(ADK_FVQSAM).First,high-level features are extracted using the DenseNet121 backbone network,followed by adaptive average pooling to scale the features to a fixed length.Subsequently,product quantization with residuals(PQR)is applied to convert continuous feature vectors into discrete features representations,preserving essential information insensitive to image quality variations.The quantized and original features are concatenated and fed into a self-attention mechanism to capture keratitis-related features.Finally,these enhanced features are classified through a fully connected layer.Experiments on clinical low-quality(LQ)images show that ADK_FVQSAM achieves accuracies of 87.7%,81.9%,and 89.3% for keratitis,other corneal abnormalities,and normal corneas,respectively.Compared to DenseNet121,Swin transformer,and InceptionResNet,ADK_FVQSAM improves average accuracy by 3.1%,11.3%,and 15.3%,respectively.These results demonstrate that ADK_FVQSAM significantly enhances the recognition performance of keratitis based on LQ slit-lamp images,offering a practical approach for clinical application.
基金supported in part by the National Natural Science Foundation of China under Grant 62302475in part by the Research Funds of Centre for Leading Medicine and Advanced Technologies of IHM under Grant 2023IHM01081 and 2023IHM01085+1 种基金in part by the Hefei Municipal Natural Science Foundation under Grant 202328partly by the Anhui Science and Technology Innovation Tackling Plan Project under Grant 202423k09020044。
文摘The Internet of Things(IoT)technology provides data acquisition,transmission,and analysis to control rehabilitation robots,encompassing sensor data from the robots as well as lidar signals for trajectory planning(desired trajectory).In IoT rehabilitation robot systems,managing nonvanishing uncertainties and input quantization is crucial for precise and reliable control performance.These challenges can cause instability and reduced effectiveness,particularly in adaptive networked control.This paper investigates networked control with guaranteed performance for IoT rehabilitation robots under nonvanishing uncertainties and input quantization.First,input quantization is managed via a quantization-aware control design,ensur stability and minimizing tracking errors,even with discrete control inputs,to avoid chattering.Second,the method handles nonvanishing uncertainties by adjusting control parameters via real-time neural network adaptation,maintaining consistent performance despite persistent disturbances.Third,the control scheme guarantees the desired tracking performance within a specified time,with all signals in the closed-loop system remaining uniformly bounded,offering a robust,reliable solution for IoT rehabilitation robot control.The simulation verifies the benefits and efficacy of the proposed control strategy.
文摘Formal state space models of quantum control systems are deduced and a scheme to establish formal state space models via quantization could been obtained for quantum control systems is proposed. State evolution of quantum control systems must accord with Schrdinger equations, so it is foremost to obtain Hamiltonian operators of systems. There are corresponding relations between operators of quantum systems and corresponding physical quantities of classical systems, such as momentum, energy and Hamiltonian, so Schrdinger equation models of corresponding quantum control systems via quantization could been obtained from classical control systems, and then establish formal state space models through the suitable transformation from Schrdinger equations for these quantum control systems. This method provides a new kind of path for modeling in quantum control.
基金the National Natural Science Foundation of China (60602057)the NaturalScience Foundation of Chongqing Science and Technology Commission (2006BB2373).
文摘A fast encoding algorithm based on the mean square error （MSE） distortion for vector quantization is introduced. The vector, which is effectively constructed with wavelet transform （WT） coefficients of images, can simplify the realization of the non-linear interpolated vector quantization （NLIVQ） technique and make the partial distance search （PDS） algorithm more efficient. Utilizing the relationship of vector L2-norm and its Euclidean distance, some conditions of eliminating unnecessary codewords are obtained. Further, using inequality constructed by the subvector L2-norm, more unnecessary codewords are eliminated. During the search process for code, mostly unlikely codewords can be rejected by the proposed algorithm combined with the non-linear interpolated vector quantization technique and the partial distance search technique. The experimental results show that the reduction of computation is outstanding in the encoding time and complexity against the full search method.
基金supported by the National Natural Science Foundation of China under Grant No.60133012 and No.661272374
文摘Information hiding schemes based on vector quantization （VQ） usually require lengthy VQ encoding and decoding processes. In this paper, we propose an efficient information hiding method based on closest paired tree structure vector quantization （CPTSVQ）. The simulation result shows that the execution time of the proposed scheme is much shorter than that attained by previous approaches.
文摘合法通信双方可以利用无线信道状态的随机特性提取符合信息论安全的共享密钥序列。为了提高无线信道提取密钥的效率,提出一种基于差分的多级量化共享密钥提取方案。该方案采用随机调制对无线信道进行高频采样,并引入融合随机抽样差分的自适应符号量化(ASQ)和均衡化多比特修正量化(BMMQ)这2个算法处理一阶差分序列,以获得原始密钥序列。在此基础上,应用信息协商算法纠正原始密钥中不一致的比特,并使用原始密钥及一阶差分序列重构信号,再对该信号进行二次量化,最终实现合法通信双方的密钥同步。实验结果表明,随机抽样差分能够将相邻样本点之间的相关系数降低至e^(-1)以下,有效降低密钥序列中的统计依赖性;在信噪比(SNR)为25 dB的条件下,ASQ算法可在保持原始密钥提取率(OKER)为0.86的同时,将密钥不一致率(KDR)降低至3.8×10^(-5);在无损量化的条件下,BMMQ算法可以把KDR降低至7×10^(-3)。最终生成的共享密钥序列通过了NIST(National Institute of Standards and Technology)随机性测试,验证了密钥的安全性和有效性。
文摘随着大语言模型(large language model,LLM)参数规模的指数级增长,模型部署和推理面临着严峻的内存和计算资源挑战。量化技术作为模型压缩的核心方法,通过降低权重和激活值的数值精度,显著减少了模型的存储需求和计算开销。首先回顾了量化技术的发展历程,从经典的Int8/4量化方法到前沿的超低比特量化算法,总结了典型方法的技术特征与性能演进规律,指出传统实数域量化在极低比特条件下存在受限于离散化误差的挑战,难以突破性能上限。为此,进而系统性地梳理了复域量化系列工作。该系列工作提出了基于复数域的量化范式,通过在参数表示中引入幅度与相位2个自由度,显著扩展了模型的表达空间;此外,类比信号处理中通过将时域信号进行傅里叶变换与低通滤波实现稳定表示的经典范式,进一步提出了由实数模型经复域变换与复域量化,达成了无乘法稳定推理的技术路线。实验结果表明,该方案在多个基准数据集上优于现有超低比特量化方法,有效突破了实数域模型的性能天花板,展现出复域量化在高效建模与性能保持方面的潜在价值。总体而言,通过对量化技术演进及复域量化系列研究的系统分析,旨在揭示超低比特量化的发展规律与未来趋势,为高效大模型的理论研究与工程实现提供参考。
