Large-scale Language Models(LLMs)have achieved significant breakthroughs in Natural Language Processing(NLP),driven by the pre-training and fine-tuning paradigm.While this approach allows models to specialize in speci...Large-scale Language Models(LLMs)have achieved significant breakthroughs in Natural Language Processing(NLP),driven by the pre-training and fine-tuning paradigm.While this approach allows models to specialize in specific tasks with reduced training costs,the substantial memory requirements during fine-tuning present a barrier to broader deployment.Parameter-Efficient Fine-Tuning(PEFT)techniques,such as Low-Rank Adaptation(LoRA),and parameter quantization methods have emerged as solutions to address these challenges by optimizing memory usage and computational efficiency.Among these,QLoRA,which combines PEFT and quantization,has demonstrated notable success in reducing memory footprints during fine-tuning,prompting the development of various QLoRA variants.Despite these advancements,the quantitative impact of key variables on the fine-tuning performance of quantized LLMs remains underexplored.This study presents a comprehensive analysis of these key variables,focusing on their influence across different layer types and depths within LLM architectures.Our investigation uncovers several critical findings:(1)Larger layers,such as MLP layers,can maintain performance despite reductions in adapter rank,while smaller layers,like self-attention layers,aremore sensitive to such changes;(2)The effectiveness of balancing factors depends more on specific values rather than layer type or depth;(3)In quantization-aware fine-tuning,larger layers can effectively utilize smaller adapters,whereas smaller layers struggle to do so.These insights suggest that layer type is a more significant determinant of fine-tuning success than layer depth when optimizing quantized LLMs.Moreover,for the same discount of trainable parameters,reducing the trainable parameters in a larger layer is more effective in preserving fine-tuning accuracy than in a smaller one.This study provides valuable guidance for more efficient fine-tuning strategies and opens avenues for further research into optimizing LLM fine-tuning in resource-constrained environments.展开更多
The rapid growth of digital data necessitates advanced natural language processing(NLP)models like BERT(Bidi-rectional Encoder Representations from Transformers),known for its superior performance in text classificati...The rapid growth of digital data necessitates advanced natural language processing(NLP)models like BERT(Bidi-rectional Encoder Representations from Transformers),known for its superior performance in text classification.However,BERT’s size and computational demands limit its practicality,especially in resource-constrained settings.This research compresses the BERT base model for Bengali emotion classification through knowledge distillation(KD),pruning,and quantization techniques.Despite Bengali being the sixth most spoken language globally,NLP research in this area is limited.Our approach addresses this gap by creating an efficient BERT-based model for Bengali text.We have explored 20 combinations for KD,quantization,and pruning,resulting in improved speedup,fewer parameters,and reduced memory size.Our best results demonstrate significant improvements in both speed and efficiency.For instance,in the case of mBERT,we achieved a 3.87×speedup and 4×compression ratio with a combination of Distil+Prune+Quant that reduced parameters from 178 to 46 M,while the memory size decreased from 711 to 178 MB.These results offer scalable solutions for NLP tasks in various languages and advance the field of model compression,making these models suitable for real-world applications in resource-limited environments.展开更多
We consider a relativistic two-fluid model of superfluidity,in which the superfluid is described by an order parameter that is a complex scalar field satisfying the nonlinear Klein-Gordon equation(NLKG).The coupling t...We consider a relativistic two-fluid model of superfluidity,in which the superfluid is described by an order parameter that is a complex scalar field satisfying the nonlinear Klein-Gordon equation(NLKG).The coupling to the normal fluid is introduced via a covariant current-current interaction,which results in the addition of an effective potential,whose imaginary part describes particle transfer between superfluid and normal fluid.Quantized vorticity arises in a class of singular solutions and the related vortex dynamics is incorporated in the modified NLKG,facilitating numerical analysis which is usually very complicated in the phenomenology of vortex filaments.The dual transformation to a string theory description(Kalb-Ramond)of quantum vorticity,the Magnus force,and the mutual friction between quantized vortices and normal fluid are also studied.展开更多
In this paper, we consider the design of interconnected H-infinity feedback control systems with quantized signals. We assume that a decentralized static output feedback has been designed for an interconnected continu...In this paper, we consider the design of interconnected H-infinity feedback control systems with quantized signals. We assume that a decentralized static output feedback has been designed for an interconnected continuous-time LTI system so that the closed-loop system is stable and a desired H-infinity disturbance attenuation level is achieved, and that the subsystems＇ measurement outputs are quantized before they are passed to the local controller. We propose a local-output-dependent strategy for updating the quantizers＇ parameters, so that the overall closed-loop system is asymptotically stable and achieves the same H-infinity disturbance attenuation level. Both the pre-designed controllers and the quantizers＇ parameters are constructed in a decentralized manner, depending on local information.展开更多
A new scheme is presented to design a rotated Barnes-Wall lattice based vector quantizer(LVQ). The construction method of the LVQ and its fast quantizing algorithm are described at first. Then gain-shape lattice vecto...A new scheme is presented to design a rotated Barnes-Wall lattice based vector quantizer(LVQ). The construction method of the LVQ and its fast quantizing algorithm are described at first. Then gain-shape lattice vector quantizer(GSLVQ) with LVQ as shape quantizer is discussed. Finally the GSLVQ is used in image-sequence coding and good experimental results are obtained.展开更多
This paper is concerned with distributed Nash equi librium seeking strategies under quantized communication. In the proposed seeking strategy, a projection operator is synthesized with a gradient search method to achi...This paper is concerned with distributed Nash equi librium seeking strategies under quantized communication. In the proposed seeking strategy, a projection operator is synthesized with a gradient search method to achieve the optimization o players' objective functions while restricting their actions within required non-empty, convex and compact domains. In addition, a leader-following consensus protocol, in which quantized informa tion flows are utilized, is employed for information sharing among players. More specifically, logarithmic quantizers and uniform quantizers are investigated under both undirected and connected communication graphs and strongly connected digraphs, respec tively. Through Lyapunov stability analysis, it is shown that play ers' actions can be steered to a neighborhood of the Nash equilib rium with logarithmic and uniform quantizers, and the quanti fied convergence error depends on the parameter of the quan tizer for both undirected and directed cases. A numerical exam ple is given to verify the theoretical results.展开更多
AVQ(Adaptive Vector Quantizer)overcomes some shortcomings of traditional vectorquantizer with a fixed codebook trained and generated by the LBG or other algorithms by applyinga variab|e codebook.In this paper,we descr...AVQ(Adaptive Vector Quantizer)overcomes some shortcomings of traditional vectorquantizer with a fixed codebook trained and generated by the LBG or other algorithms by applyinga variab|e codebook.In this paper,we describe an effective and efficient implementation of AVQby modifying the CCN(Carpenter/Grossberg Net).The encoding process of AVQ is very similarto the learning process of the CGN.We study several different encoding schemes,includingwaveform AVQ,analysed parameter AVQ and so on,implemented by the CGN.And we simulatethe encoding performance of each scheme for encoding Gaussian process source,first order Gauss-Markov process source and practical speech signal.Our simulation results show that good qualityboth in subjective and objective tests can be obtained in a low or middle bit rate range.展开更多
The demand for adopting neural networks in resource-constrained embedded devices is continuously increasing.Quantization is one of the most promising solutions to reduce computational cost and memory storage on embedd...The demand for adopting neural networks in resource-constrained embedded devices is continuously increasing.Quantization is one of the most promising solutions to reduce computational cost and memory storage on embedded devices.In order to reduce the complexity and overhead of deploying neural networks on Integeronly hardware,most current quantization methods use a symmetric quantization mapping strategy to quantize a floating-point neural network into an integer network.However,although symmetric quantization has the advantage of easier implementation,it is sub-optimal for cases where the range could be skewed and not symmetric.This often comes at the cost of lower accuracy.This paper proposed an activation redistribution-based hybrid asymmetric quantizationmethod for neural networks.The proposedmethod takes data distribution into consideration and can resolve the contradiction between the quantization accuracy and the ease of implementation,balance the trade-off between clipping range and quantization resolution,and thus improve the accuracy of the quantized neural network.The experimental results indicate that the accuracy of the proposed method is 2.02%and 5.52%higher than the traditional symmetric quantization method for classification and detection tasks,respectively.The proposed method paves the way for computationally intensive neural network models to be deployed on devices with limited computing resources.Codes will be available on https://github.com/ycjcy/Hybrid-Asymmetric-Quantization.展开更多
This article presents a high speed third-order continuous-time(CT)sigma-delta analog-to-digital converter(SDADC)based on voltagecontrolled oscillator(VCO),featuring a digital programmable quantizer structure.To improv...This article presents a high speed third-order continuous-time(CT)sigma-delta analog-to-digital converter(SDADC)based on voltagecontrolled oscillator(VCO),featuring a digital programmable quantizer structure.To improve the overall performance,not only oversampling technique but also noise-shaping enhancing technique is used to suppress in-band noise.Due to the intrinsic first-order noise-shaping of the VCO quantizer,the proposed third-order SDADC can realize forth-order noise-shaping ideally.As a bright advantage,the proposed programmable VCO quantizer is digital-friendly,which can simplify the design process and improve antiinterference capability of the circuit.A 4-bit programmable VCO quantizer clocked at 2.5 GHz,which is proposed in a 40 nm complementary metaloxide semiconductor(CMOS)technology,consists of an analog VCO circuit and a digital programmable quantizer,achieving 50.7 dB signal-to-noise ratio(SNR)and 26.9 dB signal-to-noise-and-distortion ration(SNDR)for a 19 MHz−3.5 dBFS input signal in 78 MHz bandwidth(BW).The digital quantizer,which is programmed in the Verilog hardware description language(HDL),consists of two-stage D-flip-flop(DFF)based registers,XOR gates and an adder.The presented SDADC adopts the cascade of integrators with feed-forward summation(CIFF)structure with a third-order loop filter,operating at 2.5 GHz and showing behavioral simulation performance of 92.9 dB SNR over 78 MHz bandwidth.展开更多
A high-speed and high-resolution optical A/D quantizer is proposed.Its architecture is discussed.Bit circuits are built by using the phase modulators in parallel.Based on the different character of the half-wave volta...A high-speed and high-resolution optical A/D quantizer is proposed.Its architecture is discussed.Bit circuits are built by using the phase modulators in parallel.Based on the different character of the half-wave voltage for every phase modulator and the polarized bias design of incident light,the RF input signal is coled and transmitted in the form of optical digital signal.According to the principle of the architecture,the high-resolution quantizers with 8-bit and 12-bit,et al.are built,which operate at 100 GS/s.Their quantization noise is invariable almost with bit circuits increasing.The simulation result of 4-bit A/D quantizer is also given.展开更多
In this tutorial paper, we explore the field of quantized feedback control, which has gained significant attention due to the growing prevalence of networked control systems. These systems require the transmission of ...In this tutorial paper, we explore the field of quantized feedback control, which has gained significant attention due to the growing prevalence of networked control systems. These systems require the transmission of feedback information, such as measurements and control signals, over digital networks, presenting novel challenges in estimation and control design. Our examination encompasses various topics, including the minimal information needed for effective feedback control, the design of quantizers, strategies for quantized control design and estimation,achieving consensus control with quantized data, and the pursuit of high-precision tracking using quantized measurements.展开更多
The quantization algorithm compresses the original network by reducing the numerical bit width of the model,which improves the computation speed. Because different layers have different redundancy and sensitivity to d...The quantization algorithm compresses the original network by reducing the numerical bit width of the model,which improves the computation speed. Because different layers have different redundancy and sensitivity to databit width. Reducing the data bit width will result in a loss of accuracy. Therefore, it is difficult to determinethe optimal bit width for different parts of the network with guaranteed accuracy. Mixed precision quantizationcan effectively reduce the amount of computation while keeping the model accuracy basically unchanged. In thispaper, a hardware-aware mixed precision quantization strategy optimal assignment algorithm adapted to low bitwidth is proposed, and reinforcement learning is used to automatically predict the mixed precision that meets theconstraints of hardware resources. In the state-space design, the standard deviation of weights is used to measurethe distribution difference of data, the execution speed feedback of simulated neural network accelerator inferenceis used as the environment to limit the action space of the agent, and the accuracy of the quantization model afterretraining is used as the reward function to guide the agent to carry out deep reinforcement learning training. Theexperimental results show that the proposed method obtains a suitable model layer-by-layer quantization strategyunder the condition that the computational resources are satisfied, and themodel accuracy is effectively improved.The proposed method has strong intelligence and certain universality and has strong application potential in thefield of mixed precision quantization and embedded neural network model deployment.展开更多
In networked system identification,how to effectively use communication resources and improve convergence speed is the focus of attention.However,there is an inherent contradiction between the two tasks.In this paper,...In networked system identification,how to effectively use communication resources and improve convergence speed is the focus of attention.However,there is an inherent contradiction between the two tasks.In this paper,the event-driven communication is used to save communication resources for the identification of finite impulse response systems,and the input design is carried out to meet the requirements of convergence speed.First,a difference-driven communication is proposed.Then,the performance of the communication mechanism is analyzed,and the calculation method of its communication rate is given.After that,according to the communication rate and the convergence rate of the identification algorithm,the input design problem is transformed into a constrained optimization problem,and the algorithm for finding the optimal solution is given.In addition,considering the case that the output is quantized by multiple thresholds,the way to calculate its communication rate is given and the influence of threshold number on communication rate is discussed.Finally,the effectiveness of the algorithm is verified by simulation.展开更多
The recently developed magic-intensity trapping technique of neutral atoms efficiently mitigates the detrimental effect of light shifts on atomic qubits and substantially enhances the coherence time. This technique re...The recently developed magic-intensity trapping technique of neutral atoms efficiently mitigates the detrimental effect of light shifts on atomic qubits and substantially enhances the coherence time. This technique relies on applying a bias magnetic field precisely parallel to the wave vector of a circularly polarized trapping laser field. However, due to the presence of the vector light shift experienced by the trapped atoms, it is challenging to precisely define a parallel magnetic field, especially at a low bias magnetic field strength, for the magic-intensity trapping of85Rb qubits. In this work, we present a method to calibrate the angle between the bias magnetic field and the trapping laser field with the compensating magnetic fields in the other two directions orthogonal to the bias magnetic field direction. Experimentally, with a constantdepth trap and a fixed bias magnetic field, we measure the respective resonant frequencies of the atomic qubits in a linearly polarized trap and a circularly polarized one via the conventional microwave Rabi spectra with different compensating magnetic fields and obtain the corresponding total magnetic fields via the respective resonant frequencies using the Breit–Rabi formula. With known total magnetic fields, the angle is a function of the other two compensating magnetic fields.Finally, the projection value of the angle on either of the directions orthogonal to the bias magnetic field direction can be reduced to 0(4)° by applying specific compensating magnetic fields. The measurement error is mainly attributed to the fluctuation of atomic temperature. Moreover, it also demonstrates that, even for a small angle, the effect is strong enough to cause large decoherence of Rabi oscillation in a magic-intensity trap. Although the compensation method demonstrated here is explored for the magic-intensity trapping technique, it can be applied to a variety of similar precision measurements with trapped neutral atoms.展开更多
In this paper,we investigate networkassisted full-duplex(NAFD)cell-free millimeter-wave(mmWave)massive multiple-input multiple-output(MIMO)systems with digital-to-analog converter(DAC)quantization and fronthaul compre...In this paper,we investigate networkassisted full-duplex(NAFD)cell-free millimeter-wave(mmWave)massive multiple-input multiple-output(MIMO)systems with digital-to-analog converter(DAC)quantization and fronthaul compression.We propose to maximize the weighted uplink and downlink sum rate by jointly optimizing the power allocation of both the transmitting remote antenna units(T-RAUs)and uplink users and the variances of the downlink and uplink fronthaul compression noises.To deal with this challenging problem,we further apply a successive convex approximation(SCA)method to handle the non-convex bidirectional limited-capacity fronthaul constraints.The simulation results verify the convergence of the proposed SCA-based algorithm and analyze the impact of fronthaul capacity and DAC quantization on the spectral efficiency of the NAFD cell-free mmWave massive MIMO systems.Moreover,some insightful conclusions are obtained through the comparisons of spectral efficiency,which shows that NAFD achieves better performance gains than cotime co-frequency full-duplex cloud radio access network(CCFD C-RAN)in the cases of practical limited-resolution DACs.Specifically,their performance gaps with 8-bit DAC quantization are larger than that with1-bit DAC quantization,which attains a 5.5-fold improvement.展开更多
Massive computational complexity and memory requirement of artificial intelligence models impede their deploy-ability on edge computing devices of the Internet of Things(IoT).While Power-of-Two(PoT)quantization is pro...Massive computational complexity and memory requirement of artificial intelligence models impede their deploy-ability on edge computing devices of the Internet of Things(IoT).While Power-of-Two(PoT)quantization is pro-posed to improve the efficiency for edge inference of Deep Neural Networks(DNNs),existing PoT schemes require a huge amount of bit-wise manipulation and have large memory overhead,and their efficiency is bounded by the bottleneck of computation latency and memory footprint.To tackle this challenge,we present an efficient inference approach on the basis of PoT quantization and model compression.An integer-only scalar PoT quantization(IOS-PoT)is designed jointly with a distribution loss regularizer,wherein the regularizer minimizes quantization errors and training disturbances.Additionally,two-stage model compression is developed to effectively reduce memory requirement,and alleviate bandwidth usage in communications of networked heterogenous learning systems.The product look-up table(P-LUT)inference scheme is leveraged to replace bit-shifting with only indexing and addition operations for achieving low-latency computation and implementing efficient edge accelerators.Finally,comprehensive experiments on Residual Networks(ResNets)and efficient architectures with Canadian Institute for Advanced Research(CIFAR),ImageNet,and Real-world Affective Faces Database(RAF-DB)datasets,indicate that our approach achieves 2×∼10×improvement in the reduction of both weight size and computation cost in comparison to state-of-the-art methods.A P-LUT accelerator prototype is implemented on the Xilinx KV260 Field Programmable Gate Array(FPGA)platform for accelerating convolution operations,with performance results showing that P-LUT reduces memory footprint by 1.45×,achieves more than 3×power efficiency and 2×resource efficiency,compared to the conventional bit-shifting scheme.展开更多
A new steganographic method by pixel-value differencing(PVD)using general quantization ranges of pixel pairs’difference values is proposed.The objective of this method is to provide a data embedding technique with a ...A new steganographic method by pixel-value differencing(PVD)using general quantization ranges of pixel pairs’difference values is proposed.The objective of this method is to provide a data embedding technique with a range table with range widths not limited to powers of 2,extending PVD-based methods to enhance their flexibility and data-embedding rates without changing their capabilities to resist security attacks.Specifically,the conventional PVD technique partitions a grayscale image into 1×2 non-overlapping blocks.The entire range[0,255]of all possible absolute values of the pixel pairs’grayscale differences in the blocks is divided into multiple quantization ranges.The width of each quantization range is a power of two to facilitate the direct embedding of the bit information with high embedding rates.Without using power-of-two range widths,the embedding rates can drop using conventional embedding techniques.In contrast,the proposed method uses general quantization range widths,and a multiple-based number conversion mechanism is employed skillfully to implement the use of nonpower-of-two range widths,with each pixel pair being employed to embed a digit in the multiple-based number.All the message bits are converted into a big multiple-based number whose digits can be embedded into the pixel pairs with a higher embedding rate.Good experimental results showed the feasibility of the proposed method and its resistance to security attacks.In addition,implementation examples are provided,where the proposed method adopts non-power-of-two range widths and employsmultiple-based number conversion to expand the data-hiding and steganalysis-resisting capabilities of other PVD methods.展开更多
The nanoscale confinement is of great important for the industrial applications of molecular sieve,desalination,and also essential in bio-logical transport systems.Massive efforts have been devoted to the influence of...The nanoscale confinement is of great important for the industrial applications of molecular sieve,desalination,and also essential in bio-logical transport systems.Massive efforts have been devoted to the influence of restricted spaces on the properties of confined fluids.However,the situation of channel-wall is crucial but attracts less attention and remains unknown.To fundamentally understand the mechanism of channel-walls in nanoconfinement,we investigated the interaction between the counter-force of the liquid and interlamellar spacing of nanochannel walls by considering the effect of both spatial confinement and surface wettability.The results reveal that the nanochannel stables at only a few discrete spacing states when its confinement is within 1.4 nm.The quantized interlayer spacing is attributed to water molecules becoming laminated structures,and the stable states are corresponding to the monolayer,bilayer and trilayer water configurations,respectively.The results can potentially help to understand the characterized interlayers spacing of graphene oxide membrane in water.Our findings are hold great promise in design of ion filtration membrane and artificial water/ion channels.展开更多
In this paper,we innovatively associate the mutual information with the frame error rate(FER)performance and propose novel quantized decoders for polar codes.Based on the optimal quantizer of binary-input discrete mem...In this paper,we innovatively associate the mutual information with the frame error rate(FER)performance and propose novel quantized decoders for polar codes.Based on the optimal quantizer of binary-input discrete memoryless channels(BDMCs),the proposed decoders quantize the virtual subchannels of polar codes to maximize mutual information(MMI)between source bits and quantized symbols.The nested structure of polar codes ensures that the MMI quantization can be implemented stage by stage.Simulation results show that the proposed MMI decoders with 4 quantization bits outperform the existing nonuniform quantized decoders that minimize mean-squared error(MMSE)with 4 quantization bits,and yield even better performance than uniform MMI quantized decoders with 5 quantization bits.Furthermore,the proposed 5-bit quantized MMI decoders approach the floating-point decoders with negligible performance loss.展开更多
基金supported by the National Key R&D Program of China(No.2021YFB0301200)National Natural Science Foundation of China(No.62025208).
文摘Large-scale Language Models(LLMs)have achieved significant breakthroughs in Natural Language Processing(NLP),driven by the pre-training and fine-tuning paradigm.While this approach allows models to specialize in specific tasks with reduced training costs,the substantial memory requirements during fine-tuning present a barrier to broader deployment.Parameter-Efficient Fine-Tuning(PEFT)techniques,such as Low-Rank Adaptation(LoRA),and parameter quantization methods have emerged as solutions to address these challenges by optimizing memory usage and computational efficiency.Among these,QLoRA,which combines PEFT and quantization,has demonstrated notable success in reducing memory footprints during fine-tuning,prompting the development of various QLoRA variants.Despite these advancements,the quantitative impact of key variables on the fine-tuning performance of quantized LLMs remains underexplored.This study presents a comprehensive analysis of these key variables,focusing on their influence across different layer types and depths within LLM architectures.Our investigation uncovers several critical findings:(1)Larger layers,such as MLP layers,can maintain performance despite reductions in adapter rank,while smaller layers,like self-attention layers,aremore sensitive to such changes;(2)The effectiveness of balancing factors depends more on specific values rather than layer type or depth;(3)In quantization-aware fine-tuning,larger layers can effectively utilize smaller adapters,whereas smaller layers struggle to do so.These insights suggest that layer type is a more significant determinant of fine-tuning success than layer depth when optimizing quantized LLMs.Moreover,for the same discount of trainable parameters,reducing the trainable parameters in a larger layer is more effective in preserving fine-tuning accuracy than in a smaller one.This study provides valuable guidance for more efficient fine-tuning strategies and opens avenues for further research into optimizing LLM fine-tuning in resource-constrained environments.
文摘The rapid growth of digital data necessitates advanced natural language processing(NLP)models like BERT(Bidi-rectional Encoder Representations from Transformers),known for its superior performance in text classification.However,BERT’s size and computational demands limit its practicality,especially in resource-constrained settings.This research compresses the BERT base model for Bengali emotion classification through knowledge distillation(KD),pruning,and quantization techniques.Despite Bengali being the sixth most spoken language globally,NLP research in this area is limited.Our approach addresses this gap by creating an efficient BERT-based model for Bengali text.We have explored 20 combinations for KD,quantization,and pruning,resulting in improved speedup,fewer parameters,and reduced memory size.Our best results demonstrate significant improvements in both speed and efficiency.For instance,in the case of mBERT,we achieved a 3.87×speedup and 4×compression ratio with a combination of Distil+Prune+Quant that reduced parameters from 178 to 46 M,while the memory size decreased from 711 to 178 MB.These results offer scalable solutions for NLP tasks in various languages and advance the field of model compression,making these models suitable for real-world applications in resource-limited environments.
文摘We consider a relativistic two-fluid model of superfluidity,in which the superfluid is described by an order parameter that is a complex scalar field satisfying the nonlinear Klein-Gordon equation(NLKG).The coupling to the normal fluid is introduced via a covariant current-current interaction,which results in the addition of an effective potential,whose imaginary part describes particle transfer between superfluid and normal fluid.Quantized vorticity arises in a class of singular solutions and the related vortex dynamics is incorporated in the modified NLKG,facilitating numerical analysis which is usually very complicated in the phenomenology of vortex filaments.The dual transformation to a string theory description(Kalb-Ramond)of quantum vorticity,the Magnus force,and the mutual friction between quantized vortices and normal fluid are also studied.
基金Supported by National Natural Science Foundation of China （60674021）, Program for New Century Excellent Talents in University （NCET-04-0283）, the Funds for Creative Research Groups of China （60521003）, Program for Changjiang Scholars and Innovative Research Team in University （IRT0421）, the State Key Program of National Natural Science of China （60534010） and the Funds of Ph. D. Program of Ministry of Education, China （20060145019）
文摘使量子化的动态产量反馈 H 控制为的问题分离时间线性时间不变(LTI ) 系统在这份报纸被调查。考虑的 quantizer 动态、镇静一可调节激增参数和静态的 quantizer。静态的 quantizer 范围具有实际意义并且充分被考虑。首先，考虑量子化错误，使量子化的控制策略控制器状态依赖于不仅而且在系统测量产量上，它被建议以便使量子化的靠近环的系统是 asymptotically 稳定的，与规定 H，性能跳。根据这结果，然后，一个反复的基于 LMI 的优化算法被开发优化静态的 quantizer 范围为靠近环的系统满足 H 表演要求。一个例子被举说明建议方法的有效性。
基金supported by the Japan Ministry of Education,Sciences and Culture under Grant-in-Aid for Scientific Research(C)(No.21560471)
文摘In this paper, we consider the design of interconnected H-infinity feedback control systems with quantized signals. We assume that a decentralized static output feedback has been designed for an interconnected continuous-time LTI system so that the closed-loop system is stable and a desired H-infinity disturbance attenuation level is achieved, and that the subsystems＇ measurement outputs are quantized before they are passed to the local controller. We propose a local-output-dependent strategy for updating the quantizers＇ parameters, so that the overall closed-loop system is asymptotically stable and achieves the same H-infinity disturbance attenuation level. Both the pre-designed controllers and the quantizers＇ parameters are constructed in a decentralized manner, depending on local information.
基金Supported in part by subject 863-317 (China Communication 863 Programme)Fund of Xidian University and ISN National Key Lab
文摘A new scheme is presented to design a rotated Barnes-Wall lattice based vector quantizer(LVQ). The construction method of the LVQ and its fast quantizing algorithm are described at first. Then gain-shape lattice vector quantizer(GSLVQ) with LVQ as shape quantizer is discussed. Finally the GSLVQ is used in image-sequence coding and good experimental results are obtained.
基金supported by the National Natural Science Foundation of China (NSFC)(62222308, 62173181, 62073171, 62221004)the Natural Science Foundation of Jiangsu Province (BK20200744, BK20220139)+3 种基金Jiangsu Specially-Appointed Professor (RK043STP19001)the Young Elite Scientists Sponsorship Program by CAST (2021QNRC001)1311 Talent Plan of Nanjing University of Posts and Telecommunicationsthe Fundamental Research Funds for the Central Universities (30920032203)。
文摘This paper is concerned with distributed Nash equi librium seeking strategies under quantized communication. In the proposed seeking strategy, a projection operator is synthesized with a gradient search method to achieve the optimization o players' objective functions while restricting their actions within required non-empty, convex and compact domains. In addition, a leader-following consensus protocol, in which quantized informa tion flows are utilized, is employed for information sharing among players. More specifically, logarithmic quantizers and uniform quantizers are investigated under both undirected and connected communication graphs and strongly connected digraphs, respec tively. Through Lyapunov stability analysis, it is shown that play ers' actions can be steered to a neighborhood of the Nash equilib rium with logarithmic and uniform quantizers, and the quanti fied convergence error depends on the parameter of the quan tizer for both undirected and directed cases. A numerical exam ple is given to verify the theoretical results.
文摘AVQ(Adaptive Vector Quantizer)overcomes some shortcomings of traditional vectorquantizer with a fixed codebook trained and generated by the LBG or other algorithms by applyinga variab|e codebook.In this paper,we describe an effective and efficient implementation of AVQby modifying the CCN(Carpenter/Grossberg Net).The encoding process of AVQ is very similarto the learning process of the CGN.We study several different encoding schemes,includingwaveform AVQ,analysed parameter AVQ and so on,implemented by the CGN.And we simulatethe encoding performance of each scheme for encoding Gaussian process source,first order Gauss-Markov process source and practical speech signal.Our simulation results show that good qualityboth in subjective and objective tests can be obtained in a low or middle bit rate range.
基金The Qian Xuesen Youth Innovation Foundation from China Aerospace Science and Technology Corporation(Grant Number 2022JY51).
文摘The demand for adopting neural networks in resource-constrained embedded devices is continuously increasing.Quantization is one of the most promising solutions to reduce computational cost and memory storage on embedded devices.In order to reduce the complexity and overhead of deploying neural networks on Integeronly hardware,most current quantization methods use a symmetric quantization mapping strategy to quantize a floating-point neural network into an integer network.However,although symmetric quantization has the advantage of easier implementation,it is sub-optimal for cases where the range could be skewed and not symmetric.This often comes at the cost of lower accuracy.This paper proposed an activation redistribution-based hybrid asymmetric quantizationmethod for neural networks.The proposedmethod takes data distribution into consideration and can resolve the contradiction between the quantization accuracy and the ease of implementation,balance the trade-off between clipping range and quantization resolution,and thus improve the accuracy of the quantized neural network.The experimental results indicate that the accuracy of the proposed method is 2.02%and 5.52%higher than the traditional symmetric quantization method for classification and detection tasks,respectively.The proposed method paves the way for computationally intensive neural network models to be deployed on devices with limited computing resources.Codes will be available on https://github.com/ycjcy/Hybrid-Asymmetric-Quantization.
基金This work was supported by the Natural Science Foundation of the Jiangsu Higher Education Institutions of China under Grant No.18KJB510045.
文摘This article presents a high speed third-order continuous-time(CT)sigma-delta analog-to-digital converter(SDADC)based on voltagecontrolled oscillator(VCO),featuring a digital programmable quantizer structure.To improve the overall performance,not only oversampling technique but also noise-shaping enhancing technique is used to suppress in-band noise.Due to the intrinsic first-order noise-shaping of the VCO quantizer,the proposed third-order SDADC can realize forth-order noise-shaping ideally.As a bright advantage,the proposed programmable VCO quantizer is digital-friendly,which can simplify the design process and improve antiinterference capability of the circuit.A 4-bit programmable VCO quantizer clocked at 2.5 GHz,which is proposed in a 40 nm complementary metaloxide semiconductor(CMOS)technology,consists of an analog VCO circuit and a digital programmable quantizer,achieving 50.7 dB signal-to-noise ratio(SNR)and 26.9 dB signal-to-noise-and-distortion ration(SNDR)for a 19 MHz−3.5 dBFS input signal in 78 MHz bandwidth(BW).The digital quantizer,which is programmed in the Verilog hardware description language(HDL),consists of two-stage D-flip-flop(DFF)based registers,XOR gates and an adder.The presented SDADC adopts the cascade of integrators with feed-forward summation(CIFF)structure with a third-order loop filter,operating at 2.5 GHz and showing behavioral simulation performance of 92.9 dB SNR over 78 MHz bandwidth.
基金Natural Science Foundation from Colleges and Universities of Jiangsu Province(04KJD140033)
文摘A high-speed and high-resolution optical A/D quantizer is proposed.Its architecture is discussed.Bit circuits are built by using the phase modulators in parallel.Based on the different character of the half-wave voltage for every phase modulator and the polarized bias design of incident light,the RF input signal is coled and transmitted in the form of optical digital signal.According to the principle of the architecture,the high-resolution quantizers with 8-bit and 12-bit,et al.are built,which operate at 100 GS/s.Their quantization noise is invariable almost with bit circuits increasing.The simulation result of 4-bit A/D quantizer is also given.
基金partially supported by National Natura Science Foundation of China (62350710214, U23A20325)Shenzhen Key Laboratory of Control Theory and Intelligent Systems (ZDSYS20220330161800001)。
文摘In this tutorial paper, we explore the field of quantized feedback control, which has gained significant attention due to the growing prevalence of networked control systems. These systems require the transmission of feedback information, such as measurements and control signals, over digital networks, presenting novel challenges in estimation and control design. Our examination encompasses various topics, including the minimal information needed for effective feedback control, the design of quantizers, strategies for quantized control design and estimation,achieving consensus control with quantized data, and the pursuit of high-precision tracking using quantized measurements.
文摘The quantization algorithm compresses the original network by reducing the numerical bit width of the model,which improves the computation speed. Because different layers have different redundancy and sensitivity to databit width. Reducing the data bit width will result in a loss of accuracy. Therefore, it is difficult to determinethe optimal bit width for different parts of the network with guaranteed accuracy. Mixed precision quantizationcan effectively reduce the amount of computation while keeping the model accuracy basically unchanged. In thispaper, a hardware-aware mixed precision quantization strategy optimal assignment algorithm adapted to low bitwidth is proposed, and reinforcement learning is used to automatically predict the mixed precision that meets theconstraints of hardware resources. In the state-space design, the standard deviation of weights is used to measurethe distribution difference of data, the execution speed feedback of simulated neural network accelerator inferenceis used as the environment to limit the action space of the agent, and the accuracy of the quantization model afterretraining is used as the reward function to guide the agent to carry out deep reinforcement learning training. Theexperimental results show that the proposed method obtains a suitable model layer-by-layer quantization strategyunder the condition that the computational resources are satisfied, and themodel accuracy is effectively improved.The proposed method has strong intelligence and certain universality and has strong application potential in thefield of mixed precision quantization and embedded neural network model deployment.
基金supported in part by the National Natural Science Foundation of China(No.62173030)in part by the Beijing Natural Science Foundation(No.4222050).
文摘In networked system identification,how to effectively use communication resources and improve convergence speed is the focus of attention.However,there is an inherent contradiction between the two tasks.In this paper,the event-driven communication is used to save communication resources for the identification of finite impulse response systems,and the input design is carried out to meet the requirements of convergence speed.First,a difference-driven communication is proposed.Then,the performance of the communication mechanism is analyzed,and the calculation method of its communication rate is given.After that,according to the communication rate and the convergence rate of the identification algorithm,the input design problem is transformed into a constrained optimization problem,and the algorithm for finding the optimal solution is given.In addition,considering the case that the output is quantized by multiple thresholds,the way to calculate its communication rate is given and the influence of threshold number on communication rate is discussed.Finally,the effectiveness of the algorithm is verified by simulation.
基金Project supported by the National Natural Science Foundation of China(Grant Nos.12104414,12122412,12104464,and 12104413)the China Postdoctoral Science Foundation(Grant No.2021M702955).
文摘The recently developed magic-intensity trapping technique of neutral atoms efficiently mitigates the detrimental effect of light shifts on atomic qubits and substantially enhances the coherence time. This technique relies on applying a bias magnetic field precisely parallel to the wave vector of a circularly polarized trapping laser field. However, due to the presence of the vector light shift experienced by the trapped atoms, it is challenging to precisely define a parallel magnetic field, especially at a low bias magnetic field strength, for the magic-intensity trapping of85Rb qubits. In this work, we present a method to calibrate the angle between the bias magnetic field and the trapping laser field with the compensating magnetic fields in the other two directions orthogonal to the bias magnetic field direction. Experimentally, with a constantdepth trap and a fixed bias magnetic field, we measure the respective resonant frequencies of the atomic qubits in a linearly polarized trap and a circularly polarized one via the conventional microwave Rabi spectra with different compensating magnetic fields and obtain the corresponding total magnetic fields via the respective resonant frequencies using the Breit–Rabi formula. With known total magnetic fields, the angle is a function of the other two compensating magnetic fields.Finally, the projection value of the angle on either of the directions orthogonal to the bias magnetic field direction can be reduced to 0(4)° by applying specific compensating magnetic fields. The measurement error is mainly attributed to the fluctuation of atomic temperature. Moreover, it also demonstrates that, even for a small angle, the effect is strong enough to cause large decoherence of Rabi oscillation in a magic-intensity trap. Although the compensation method demonstrated here is explored for the magic-intensity trapping technique, it can be applied to a variety of similar precision measurements with trapped neutral atoms.
基金supported in part by the National Natural Science Foundation of China(NSFC)under Grants 61971127,61871465,61871122in part by the National Key Research and Development Program under Grant 2020YFB1806600in part by the open research fund of National Mobile Communications Research Laboratory,Southeast University under Grant 2022D11。
文摘In this paper,we investigate networkassisted full-duplex(NAFD)cell-free millimeter-wave(mmWave)massive multiple-input multiple-output(MIMO)systems with digital-to-analog converter(DAC)quantization and fronthaul compression.We propose to maximize the weighted uplink and downlink sum rate by jointly optimizing the power allocation of both the transmitting remote antenna units(T-RAUs)and uplink users and the variances of the downlink and uplink fronthaul compression noises.To deal with this challenging problem,we further apply a successive convex approximation(SCA)method to handle the non-convex bidirectional limited-capacity fronthaul constraints.The simulation results verify the convergence of the proposed SCA-based algorithm and analyze the impact of fronthaul capacity and DAC quantization on the spectral efficiency of the NAFD cell-free mmWave massive MIMO systems.Moreover,some insightful conclusions are obtained through the comparisons of spectral efficiency,which shows that NAFD achieves better performance gains than cotime co-frequency full-duplex cloud radio access network(CCFD C-RAN)in the cases of practical limited-resolution DACs.Specifically,their performance gaps with 8-bit DAC quantization are larger than that with1-bit DAC quantization,which attains a 5.5-fold improvement.
基金This work was supported by Open Fund Project of State Key Laboratory of Intelligent Vehicle Safety Technology by Grant with No.IVSTSKL-202311Key Projects of Science and Technology Research Programme of Chongqing Municipal Education Commission by Grant with No.KJZD-K202301505+1 种基金Cooperation Project between Chongqing Municipal Undergraduate Universities and Institutes Affiliated to the Chinese Academy of Sciences in 2021 by Grant with No.HZ2021015Chongqing Graduate Student Research Innovation Program by Grant with No.CYS240801.
文摘Massive computational complexity and memory requirement of artificial intelligence models impede their deploy-ability on edge computing devices of the Internet of Things(IoT).While Power-of-Two(PoT)quantization is pro-posed to improve the efficiency for edge inference of Deep Neural Networks(DNNs),existing PoT schemes require a huge amount of bit-wise manipulation and have large memory overhead,and their efficiency is bounded by the bottleneck of computation latency and memory footprint.To tackle this challenge,we present an efficient inference approach on the basis of PoT quantization and model compression.An integer-only scalar PoT quantization(IOS-PoT)is designed jointly with a distribution loss regularizer,wherein the regularizer minimizes quantization errors and training disturbances.Additionally,two-stage model compression is developed to effectively reduce memory requirement,and alleviate bandwidth usage in communications of networked heterogenous learning systems.The product look-up table(P-LUT)inference scheme is leveraged to replace bit-shifting with only indexing and addition operations for achieving low-latency computation and implementing efficient edge accelerators.Finally,comprehensive experiments on Residual Networks(ResNets)and efficient architectures with Canadian Institute for Advanced Research(CIFAR),ImageNet,and Real-world Affective Faces Database(RAF-DB)datasets,indicate that our approach achieves 2×∼10×improvement in the reduction of both weight size and computation cost in comparison to state-of-the-art methods.A P-LUT accelerator prototype is implemented on the Xilinx KV260 Field Programmable Gate Array(FPGA)platform for accelerating convolution operations,with performance results showing that P-LUT reduces memory footprint by 1.45×,achieves more than 3×power efficiency and 2×resource efficiency,compared to the conventional bit-shifting scheme.
文摘A new steganographic method by pixel-value differencing(PVD)using general quantization ranges of pixel pairs’difference values is proposed.The objective of this method is to provide a data embedding technique with a range table with range widths not limited to powers of 2,extending PVD-based methods to enhance their flexibility and data-embedding rates without changing their capabilities to resist security attacks.Specifically,the conventional PVD technique partitions a grayscale image into 1×2 non-overlapping blocks.The entire range[0,255]of all possible absolute values of the pixel pairs’grayscale differences in the blocks is divided into multiple quantization ranges.The width of each quantization range is a power of two to facilitate the direct embedding of the bit information with high embedding rates.Without using power-of-two range widths,the embedding rates can drop using conventional embedding techniques.In contrast,the proposed method uses general quantization range widths,and a multiple-based number conversion mechanism is employed skillfully to implement the use of nonpower-of-two range widths,with each pixel pair being employed to embed a digit in the multiple-based number.All the message bits are converted into a big multiple-based number whose digits can be embedded into the pixel pairs with a higher embedding rate.Good experimental results showed the feasibility of the proposed method and its resistance to security attacks.In addition,implementation examples are provided,where the proposed method adopts non-power-of-two range widths and employsmultiple-based number conversion to expand the data-hiding and steganalysis-resisting capabilities of other PVD methods.
基金support from the National Natural Science Foundation of China(Grant Nos.12372327,12372109,11972171)National Key R&D Program of China(Grant No.2023YFB4605101).
文摘The nanoscale confinement is of great important for the industrial applications of molecular sieve,desalination,and also essential in bio-logical transport systems.Massive efforts have been devoted to the influence of restricted spaces on the properties of confined fluids.However,the situation of channel-wall is crucial but attracts less attention and remains unknown.To fundamentally understand the mechanism of channel-walls in nanoconfinement,we investigated the interaction between the counter-force of the liquid and interlamellar spacing of nanochannel walls by considering the effect of both spatial confinement and surface wettability.The results reveal that the nanochannel stables at only a few discrete spacing states when its confinement is within 1.4 nm.The quantized interlayer spacing is attributed to water molecules becoming laminated structures,and the stable states are corresponding to the monolayer,bilayer and trilayer water configurations,respectively.The results can potentially help to understand the characterized interlayers spacing of graphene oxide membrane in water.Our findings are hold great promise in design of ion filtration membrane and artificial water/ion channels.
基金financially supported in part by National Key R&D Program of China(No.2018YFB1801402)in part by Huawei Technologies Co.,Ltd.
文摘In this paper,we innovatively associate the mutual information with the frame error rate(FER)performance and propose novel quantized decoders for polar codes.Based on the optimal quantizer of binary-input discrete memoryless channels(BDMCs),the proposed decoders quantize the virtual subchannels of polar codes to maximize mutual information(MMI)between source bits and quantized symbols.The nested structure of polar codes ensures that the MMI quantization can be implemented stage by stage.Simulation results show that the proposed MMI decoders with 4 quantization bits outperform the existing nonuniform quantized decoders that minimize mean-squared error(MMSE)with 4 quantization bits,and yield even better performance than uniform MMI quantized decoders with 5 quantization bits.Furthermore,the proposed 5-bit quantized MMI decoders approach the floating-point decoders with negligible performance loss.
