This communique is opted to study the approximate solution of the Algebraic Lyapunov equation on the manifold of positive-definite Hermitian matrices.We choose the geodesic distance betweenAHXXA an...This communique is opted to study the approximate solution of the Algebraic Lyapunov equation on the manifold of positive-definite Hermitian matrices.We choose the geodesic distance betweenAHXXA and P as the cost function,and put forward the Extended Hamiltonian algorithm(EHA)and Natural gradient algorithm(NGA)for the solution.Finally,several numerical experiments give you an idea about the effectiveness of the proposed algorithms.We also show the comparison between these two algorithms EHA and NGA.Obtained results are provided and analyzed graphically.We also conclude that the extended Hamiltonian algorithm has better convergence speed than the natural gradient algorithm,whereas the trajectory of the solution matrix is optimal in case of Natural gradient algorithm(NGA)as compared to Extended Hamiltonian Algorithm(EHA).The aim of this paper is to show that the Extended Hamiltonian algorithm(EHA)has superior convergence properties as compared to Natural gradient algorithm(NGA).Upto the best of author’s knowledge,no approximate solution of the Algebraic Lyapunov equation on the manifold of positive-definite Hermitian matrices is found so far in the literature.展开更多
在雷达通信一体化领域,设计出既能实现雷达探测功能又能实现通信信息传输功能的同波形信号是至关重要的一个环节。针对在雷达信号脉冲内对通信信息调制后自相关性能低的问题,提出一种高频带利用率以及低自相关旁瓣的基于非线性调频(NLFM...在雷达通信一体化领域,设计出既能实现雷达探测功能又能实现通信信息传输功能的同波形信号是至关重要的一个环节。针对在雷达信号脉冲内对通信信息调制后自相关性能低的问题,提出一种高频带利用率以及低自相关旁瓣的基于非线性调频(NLFM)信号的雷达通信一体化信号形式。将NLFM信号作为16阶正交幅度调制(16QAM)信号的载波,建立NLFM-16QAM雷达通信一体化信号模型,分析该信号的模糊函数以及相关的雷达与通信性能。在此基础上,针对所提出的NLFM-16QAM信号因其通信基带信号的随机性使雷达功能受到影响,从而降低了运动目标探测性能这一问题,将一体化系统的接收端作出改进,提出小波包降噪联合自然梯度算法对NLFM-16QAM信号进行接收处理。仿真结果表明,所提信号的频带利用率明显高于低阶调制的雷达通信一体化信号的频带利用率,在自相关性能方面,所提信号比16QAM-LFM信号的积分旁瓣比降低了23.07 d B,峰值旁瓣比降低了26.08 d B,NLFM-16QAM信号在经过改进接收端的联合算法处理后,运动目标的检测结果获得显著改善。展开更多
A new framework based on the curved Riemannian manifold is proposed to calculate the numerical solution of the Lyapunov matrix equation by using a natural gradient descent algorithm and taking the geodesic distance as...A new framework based on the curved Riemannian manifold is proposed to calculate the numerical solution of the Lyapunov matrix equation by using a natural gradient descent algorithm and taking the geodesic distance as the objective function. Moreover, a gradient descent algorithm based on the classical Euclidean distance is provided to compare with this natural gradient descent algorithm. Furthermore, the behaviors of two proposed algorithms and the conventional modified conjugate gradient algorithm are compared and demonstrated by two simulation examples. By comparison, it is shown that the convergence speed of the natural gradient descent algorithm is faster than both of the gradient descent algorithm and the conventional modified conjugate gradient algorithm in solving the Lyapunov equation.展开更多
行动器评判器(Actor Critic,简称AC)算法是强化学习连续动作领域的一类重要算法,其采用独立的结构表示策略,但更新策略时需要大量样本导致样本效率不高.为了解决该问题,提出了基于模型学习和经验回放加速的正则化自然AC算法(Regularized...行动器评判器(Actor Critic,简称AC)算法是强化学习连续动作领域的一类重要算法,其采用独立的结构表示策略,但更新策略时需要大量样本导致样本效率不高.为了解决该问题,提出了基于模型学习和经验回放加速的正则化自然AC算法(Regularized Natural AC with Model Learning and Experience Replay,简称RNAC-ML-ER).RNAC-ML-ER将Agent与环境在线交互产生的样本用于学习系统动态性对应的线性模型和填充经验回放存储器.将线性模型产生的模拟样本和经验回放存储器中存储的样本作为在线样本的补充,实现值函数、优势函数和策略的更新.为了提高更新的效率,在每个时间步,仅当模型的预测误差未超过阈值时才利用该模型进行规划,同时根据TD-error从大到小的顺序对经验回放存储器中的样本进行回放.为了降低策略梯度估计的方差,引入优势函数参数向量对优势函数进行线性近似,在优势函数的目标函数中加入2-范数进行正则化,并通过优势函数参数向量来对策略梯度更新,以促进优势函数和策略的收敛.在指定的两个假设成立的条件下,通过理论分析证明了所提算法RNAC-ML-ER的收敛性.在4个强化学习的经典问题即平衡杆、小车上山、倒立摆和体操机器人中对RNACML-ER算法进行实验,结果表明所提算法能在大幅提高样本效率和学习速率的同时保持较高的稳定性.展开更多
文摘This communique is opted to study the approximate solution of the Algebraic Lyapunov equation on the manifold of positive-definite Hermitian matrices.We choose the geodesic distance betweenAHXXA and P as the cost function,and put forward the Extended Hamiltonian algorithm(EHA)and Natural gradient algorithm(NGA)for the solution.Finally,several numerical experiments give you an idea about the effectiveness of the proposed algorithms.We also show the comparison between these two algorithms EHA and NGA.Obtained results are provided and analyzed graphically.We also conclude that the extended Hamiltonian algorithm has better convergence speed than the natural gradient algorithm,whereas the trajectory of the solution matrix is optimal in case of Natural gradient algorithm(NGA)as compared to Extended Hamiltonian Algorithm(EHA).The aim of this paper is to show that the Extended Hamiltonian algorithm(EHA)has superior convergence properties as compared to Natural gradient algorithm(NGA).Upto the best of author’s knowledge,no approximate solution of the Algebraic Lyapunov equation on the manifold of positive-definite Hermitian matrices is found so far in the literature.
文摘在雷达通信一体化领域,设计出既能实现雷达探测功能又能实现通信信息传输功能的同波形信号是至关重要的一个环节。针对在雷达信号脉冲内对通信信息调制后自相关性能低的问题,提出一种高频带利用率以及低自相关旁瓣的基于非线性调频(NLFM)信号的雷达通信一体化信号形式。将NLFM信号作为16阶正交幅度调制(16QAM)信号的载波,建立NLFM-16QAM雷达通信一体化信号模型,分析该信号的模糊函数以及相关的雷达与通信性能。在此基础上,针对所提出的NLFM-16QAM信号因其通信基带信号的随机性使雷达功能受到影响,从而降低了运动目标探测性能这一问题,将一体化系统的接收端作出改进,提出小波包降噪联合自然梯度算法对NLFM-16QAM信号进行接收处理。仿真结果表明,所提信号的频带利用率明显高于低阶调制的雷达通信一体化信号的频带利用率,在自相关性能方面,所提信号比16QAM-LFM信号的积分旁瓣比降低了23.07 d B,峰值旁瓣比降低了26.08 d B,NLFM-16QAM信号在经过改进接收端的联合算法处理后,运动目标的检测结果获得显著改善。
文摘A new framework based on the curved Riemannian manifold is proposed to calculate the numerical solution of the Lyapunov matrix equation by using a natural gradient descent algorithm and taking the geodesic distance as the objective function. Moreover, a gradient descent algorithm based on the classical Euclidean distance is provided to compare with this natural gradient descent algorithm. Furthermore, the behaviors of two proposed algorithms and the conventional modified conjugate gradient algorithm are compared and demonstrated by two simulation examples. By comparison, it is shown that the convergence speed of the natural gradient descent algorithm is faster than both of the gradient descent algorithm and the conventional modified conjugate gradient algorithm in solving the Lyapunov equation.
文摘行动器评判器(Actor Critic,简称AC)算法是强化学习连续动作领域的一类重要算法,其采用独立的结构表示策略,但更新策略时需要大量样本导致样本效率不高.为了解决该问题,提出了基于模型学习和经验回放加速的正则化自然AC算法(Regularized Natural AC with Model Learning and Experience Replay,简称RNAC-ML-ER).RNAC-ML-ER将Agent与环境在线交互产生的样本用于学习系统动态性对应的线性模型和填充经验回放存储器.将线性模型产生的模拟样本和经验回放存储器中存储的样本作为在线样本的补充,实现值函数、优势函数和策略的更新.为了提高更新的效率,在每个时间步,仅当模型的预测误差未超过阈值时才利用该模型进行规划,同时根据TD-error从大到小的顺序对经验回放存储器中的样本进行回放.为了降低策略梯度估计的方差,引入优势函数参数向量对优势函数进行线性近似,在优势函数的目标函数中加入2-范数进行正则化,并通过优势函数参数向量来对策略梯度更新,以促进优势函数和策略的收敛.在指定的两个假设成立的条件下,通过理论分析证明了所提算法RNAC-ML-ER的收敛性.在4个强化学习的经典问题即平衡杆、小车上山、倒立摆和体操机器人中对RNACML-ER算法进行实验,结果表明所提算法能在大幅提高样本效率和学习速率的同时保持较高的稳定性.
