Capable of flexibly supporting diverse applications and providing computation services,the Mobile Edge Computing(MEC)-assisted Unmanned Aerial Vehicle(UAV)network is emerging as an innovational paradigm.In this paradi...Capable of flexibly supporting diverse applications and providing computation services,the Mobile Edge Computing(MEC)-assisted Unmanned Aerial Vehicle(UAV)network is emerging as an innovational paradigm.In this paradigm,the heterogeneous resources of the network,including computing and communication resources,should be allocated properly to reduce computation and communication latency as well as energy consumption.However,most existing works solely focus on the optimization issues with global information,which is generally difficult to obtain in real-world scenarios.In this paper,fully considering the incomplete information resulting from diverse types of tasks,we study the joint task offloading and spectrum allocation problem in UAV network,where free UAV nodes serve as helpers for cooperative computation.The objective is to jointly optimize offloading mode,collaboration pairing,and channel allocation to minimize the weighted network cost.To achieve the purpose with only partial observation,an extensive-form game is introduced to reformulate the problem,and a regret learning-based scheme is proposed to achieve the equilibrium solution.With retrospective improvement property and information set concept,the designed algorithm is capable of combating incomplete information and obtaining more precise allocation patterns for diverse tasks.Numerical results show that our proposed algorithm outperforms the benchmarks across various settings.展开更多
This paper considers a linear-quadratic(LQ) meanfield game governed by a forward-backward stochastic system with partial observation and common noise,where a coupling structure enters state equations,cost functionals ...This paper considers a linear-quadratic(LQ) meanfield game governed by a forward-backward stochastic system with partial observation and common noise,where a coupling structure enters state equations,cost functionals and observation equations.Firstly,to reduce the complexity of solving the meanfield game,a limiting control problem is introduced.By virtue of the decomposition approach,an admissible control set is proposed.Applying a filter technique and dimensional-expansion technique,a decentralized control strategy and a consistency condition system are derived,and the related solvability is also addressed.Secondly,we discuss an approximate Nash equilibrium property of the decentralized control strategy.Finally,we work out a financial problem with some numerical simulations.展开更多
We present a numerical approach for modeling unknown dynamical systems using partially observed data,with a focus on biological systems with(relatively)complex dynamical behavior.As an extension of the recently develo...We present a numerical approach for modeling unknown dynamical systems using partially observed data,with a focus on biological systems with(relatively)complex dynamical behavior.As an extension of the recently developed deep neural network(DNN)learning methods,our approach is particularly suitable for practical situations when(i)measurement data are available for only a subset of the state variables,and(ii)the system parameters cannot be observed or measured at all.We demonstrate that,with a properly designed DNN structure with memory terms,effective DNN models can be learned from such partially observed data containing hidden parameters.The learned DNN model serves as an accurate predictive tool for system analysis.Through a few representative biological problems,we demonstrate that such DNN models can capture qualitative dynamical behavior changes in the system,such as bifurcations,even when the parameters controlling such behavior changes are completely unknown throughout not only the model learning process but also the system prediction process.The learned DNN model effectively creates a“closed”model involving only the observables when such a closed-form model does not exist mathematically.展开更多
To cope with multi-object tracking under real-world complex situations, a new video-based method is proposed. In the detecting step, the moving objects are segmented with the third level DWT （discrete wavelet transfo...To cope with multi-object tracking under real-world complex situations, a new video-based method is proposed. In the detecting step, the moving objects are segmented with the third level DWT （discrete wavelet transform ）and background difference. In the tracking step, the Kalman filter and scale parameter are used first to estimate the object position and bounding box. Then, the center-association-based projection ratio and region-association-based occlusion ratio are defined and combined to judge object behaviours. Finally, the tracking scheme and Kalman parameters are adaptively adjusted according to object behaviour. Under occlusion, partial observability is utilized to obtain the object measurements and optimum box dimensions. This method is robust in tracking mobile objects under such situations as occlusion, new appearing and stablization, etc. Experimental results show that the proposed method is efficient.展开更多
In this paper,a deep deterministic policy gradient algorithm based on Partially Observable Weighted Mean Field Reinforcement Learning(PO-WMFRL)framework is designed to solve the problem of path planning in large-scale...In this paper,a deep deterministic policy gradient algorithm based on Partially Observable Weighted Mean Field Reinforcement Learning(PO-WMFRL)framework is designed to solve the problem of path planning in large-scale Unmanned Aerial Vehicle(UAV)swarm operations.We establish a motion control and detection communication model of UAVs.A simulation environment is carried out with No-Fly Zone(NFZ),the task assembly point is established,and the long-term reward and immediate reward functions are designed for large-scale UAV swarm path planning problem.Considering the combat characteristics of large-scale UAV swarm,we improve the traditional Deep Deterministic Policy Gradient(DDPG)algorithm and propose a Partially Observable Weighted Mean Field Deep Deterministic Policy Gradient(PO-WMFDDPG)algorithm.The effectiveness of the PO-WMFDDPG algorithm is verified through simulation,and through the comparative analysis with the DDPG and MFDDPG algorithms,it is verified that the PO-WMFDDPG algorithm has a higher task success rate and convergence speed.展开更多
A stochastic optimal control strategy for partially observable nonlinear quasi Hamiltonian systems is proposed. The optimal control forces consist of two parts. The first part is determined by the conditions under whi...A stochastic optimal control strategy for partially observable nonlinear quasi Hamiltonian systems is proposed. The optimal control forces consist of two parts. The first part is determined by the conditions under which the stochastic optimal control problem of a partially observable nonlinear system is converted into that of a completely observable linear system. The second part is determined by solving the dynamical programming equation derived by applying the stochastic averaging method and stochastic dynamical programming principle to the completely observable linear control system. The response of the optimally controlled quasi Hamiltonian system is predicted by solving the averaged Fokker-Planck-Kolmogorov equation associated with the optimally controlled completely observable linear system and solving the Riccati equation for the estimated error of system states. An example is given to illustrate the procedure and effectiveness of the proposed control strategy.展开更多
This paper investigates the fault detection problem for discrete event systems (DESs) which can be modeled by partially observed Petri nets (POPNs). To overcome the problem of low diagnosability in the POPN online fau...This paper investigates the fault detection problem for discrete event systems (DESs) which can be modeled by partially observed Petri nets (POPNs). To overcome the problem of low diagnosability in the POPN online fault diagnoser in current use, an improved online fault diagnosis algorithm that integrates generalized mutual exclusion constraints (GMECs) and integer linear programming (ILP) is proposed. Assume that the POPN structure and its initial markings are known, and the faults are modeled as unobservable transitions. First, the event sequence is observed and recorded. GMEC is used for elementary diagnosis of the system behavior, then the ILP problem of POPN is solved for further diagnosis. Finally, an example of a real DES to test the new fault diagnoser is analyzed. The proposed algorithm increases the diagnosability of the DES remarkably, and the effectiveness of the new algorithm integrating GMEC and ILP is verified.展开更多
An optimal vibration control strategy for partially observable nonlinear quasi Hamiltonian systems with actuator saturation is proposed. First,a controlled partially observable non-linear system is converted into a co...An optimal vibration control strategy for partially observable nonlinear quasi Hamiltonian systems with actuator saturation is proposed. First,a controlled partially observable non-linear system is converted into a completely observable linear control system of finite dimension based on the theorem due to Charalambous and Elliott. Then the partially averaged It stochastic differential equations and dynamical programming equation associated with the completely observable linear system are derived by using the stochastic averaging method and stochastic dynamical programming principle,respectively. The optimal control law is obtained from solving the final dynamical programming equation. The results show that the proposed control strategy has high control effectiveness and control effciency.展开更多
The authors investigate the problem of impulse control of a partially observed diffusion process. The authors study the impulse control of Zakai type equations. The associated value function is characterized as the on...The authors investigate the problem of impulse control of a partially observed diffusion process. The authors study the impulse control of Zakai type equations. The associated value function is characterized as the only viscosity solution of the corresponding quasi-variational inequality. The authors show the optimal cost function for the problem with incomplete information can be approximated by a sequence of value functions of the previous type.展开更多
A general model of insider trading on a dynamic asset in a finite time interval is proposed,in which an insider possesses the whole information on the dynamic values,noise traders without any information submit orders...A general model of insider trading on a dynamic asset in a finite time interval is proposed,in which an insider possesses the whole information on the dynamic values,noise traders without any information submit orders randomly as a martingale with volatility following a stochastic process,and market makers observe partial information when setting price in a semi-strong efficient way.With the help of filtering theory,BSDE method and dynamic programming principle,we establish a market equilibrium consisting of linear insider trading strategy and linear pricing rule,with the later characterized by price pressure on market orders and price pressure on asset observations.It shows that in the equilibrium,all the information on the risky asset is incorporated into the market price at the end of the transaction,and price pressure on market orders is a submartingale while market depth process is a martingale.Furthermore,as market makers'information precision on the asset tends to zero,the equilibrium with partial observation of market makers on the risky asset converges to the one without partial observation of market makers,while when market makers observe almost all of information on the asset,the expected profit earned by the insider makes almost zero,which is in accord with our economic intuition.Our results cover some classical results about continuous-time insider trading on a static asset.展开更多
In network service systems, satisfying quality of service (QoS) is one of the main objectives. Admission control and resource allocation strategy can be used to guarantee the QoS requirement. Based on partially observ...In network service systems, satisfying quality of service (QoS) is one of the main objectives. Admission control and resource allocation strategy can be used to guarantee the QoS requirement. Based on partially observable Markov decision processes (POMDPs), this paper proposes a novel admission control model for video on demand (VOD) service systems with elastic QoS. Elastic QoS is also considered in resource allocation strategy. Policy gradient algorithm is often available to find the solution of POMDP problems, with a satisfactory convergence rate. Through numerical examples, it can be shown that the proposed admission control strategy has better performance than complete admission control strategy.展开更多
Testing is the premise and foundation of realizing equipment health management （EHM）. To address the problem that the static periodic test strategy may cause deficient test or excessive test, a dynamic sequential te...Testing is the premise and foundation of realizing equipment health management （EHM）. To address the problem that the static periodic test strategy may cause deficient test or excessive test, a dynamic sequential test strategy （DSTS） for EHM is presented. Considering the situation that equipment health state is not completely observable in reality, a DSTS optimization method based on partially observable semi-Markov decision pro- cess （POSMDP） is proposed. Firstly, an equipment health state degradation model is constructed by Markov process, and the control limit maintenance policy is also introduced. Secondly, POSMDP is formulated in great detail. And then, POSMDP is converted to completely observable belief semi-Markov decision process （BSMDP） through belief state. The optimal equation and the corresponding optimal DSTS, which minimize the long-run ex- pected average cost per unit time, are obtained with BSMDP. The results of application in complex equipment show that the proposed DSTS is feasible and effective.展开更多
This paper examines the effect of the observation time on source identification of a discrete-time susceptible-infectedrecovered diffusion process in a network with snapshot of partial nodes.We formulate the source id...This paper examines the effect of the observation time on source identification of a discrete-time susceptible-infectedrecovered diffusion process in a network with snapshot of partial nodes.We formulate the source identification problem as a maximum likelihood(ML)estimator and develop a statistical inference method based on Monte Carlo simulation(MCS)to estimate the source location and the initial time of diffusion.Experimental results in synthetic networks and real-world networks demonstrate evident impact of the observation time as well as the fraction of the observers on the concerned problem.展开更多
Multiple earth observing satellites need to communicate with each other to observe plenty of targets on the Earth together. The factors, such as external interference, result in satellite information interaction delay...Multiple earth observing satellites need to communicate with each other to observe plenty of targets on the Earth together. The factors, such as external interference, result in satellite information interaction delays, which is unable to ensure the integrity and timeliness of the information on decision making for satellites. And the optimization of the planning result is affected. Therefore, the effect of communication delay is considered during the multi-satel ite coordinating process. For this problem, firstly, a distributed cooperative optimization problem for multiple satellites in the delayed communication environment is formulized. Secondly, based on both the analysis of the temporal sequence of tasks in a single satellite and the dynamically decoupled characteristics of the multi-satellite system, the environment information of multi-satellite distributed cooperative optimization is constructed on the basis of the directed acyclic graph（DAG）. Then, both a cooperative optimization decision making framework and a model are built according to the decentralized partial observable Markov decision process（DEC-POMDP）. After that, a satellite coordinating strategy aimed at different conditions of communication delay is mainly analyzed, and a unified processing strategy on communication delay is designed. An approximate cooperative optimization algorithm based on simulated annealing is proposed. Finally, the effectiveness and robustness of the method presented in this paper are verified via the simulation.展开更多
The transmission delay of realtime video packet mainly depends on the sensing time delay(short-term factor) and the entire frame transmission delay(long-term factor).Therefore,the optimization problem in the spectrum ...The transmission delay of realtime video packet mainly depends on the sensing time delay(short-term factor) and the entire frame transmission delay(long-term factor).Therefore,the optimization problem in the spectrum handoff process should be formulated as the combination of microscopic optimization and macroscopic optimization.In this paper,we focus on the issue of combining these two optimization models,and propose a novel Evolution Spectrum Handoff(ESH)strategy to minimize the expected transmission delay of real-time video packet.In the microoptimized model,considering the tradeoff between Primary User's(PU's) allowable collision percentage of each channel and transmission delay of video packet,we propose a mixed integer non-linear programming scheme.The scheme is able to achieve the minimum sensing time which is termed as an optimal stopping time.In the macro-optimized model,using the optimal stopping time as reward function within the partially observable Markov decision process framework,the EHS strategy is designed to search an optimal target channel set and minimize the expected delay of packet in the long-term real-time video transmission.Meanwhile,the minimum expected transmission delay is obtained under practical cognitive radio networks' conditions,i.e.,secondary user's mobility,PU's random access,imperfect sensing information,etc..Theoretical analysis and simulation results show that the ESH strategy can effectively reduce the transmission delay of video packet in spectrum handoff process.展开更多
A feedback control optimization method of partially observable linear structures via stationary response is proposed and analyzed with linear building structures equipped with control devices and sensors. First, the p...A feedback control optimization method of partially observable linear structures via stationary response is proposed and analyzed with linear building structures equipped with control devices and sensors. First, the partially observable control problem of the structure under horizontal ground acceleration excitation is converted into a completely observable control problem. Then the It6 stochastic differential equations of the system are derived based on the stochastic averaging method for quasi-integrable Hamiltonian systems and the stationary solution to the Fokker-Plank-Kolmogorov （FPK） equation associated with the It6 equations is obtained. The performance index in terms of the mean system energy and mean square control force is established and the optimal control force is obtained by minimizing the performance index. Finally, the numerical results for a three-story building structure model under E1 Centro, Hachinohe, Northridge and Kobe earthquake excitations are given to illustrate the application and the effectiveness of the proposed method.展开更多
The multi-robot systems(MRS)exploration and fire searching problem is an important application of mobile robots which require massive computation capability that exceeds the ability of traditional MRS′s.This paper pr...The multi-robot systems(MRS)exploration and fire searching problem is an important application of mobile robots which require massive computation capability that exceeds the ability of traditional MRS′s.This paper propose a cloud-based hybrid decentralized partially observable semi-Markov decision process(HDec-POSMDPs)model.The proposed model is implemented for MRS exploration and fire searching application based on the Internet of things(IoT)cloud robotics framework.In this implementation the heavy and expensive computational tasks are offloaded to the cloud servers.The proposed model achieves a significant improvement in the computation burden of the whole task relative to a traditional MRS.The proposed model is applied to explore and search for fire objects in an unknown environment;using different sets of robots sizes.The preliminary evaluation of this implementation demonstrates that as the parallelism of computational instances increase the delay of new actuation commands which will be decreased,the mean time of task completion is decreased,the number of turns in the path from the start pose cells to the target cells is minimized and the energy consumption for each robot is reduced.展开更多
A navigation method based on the partially observable markov decision process （POMDP） for smart wheelchairs in uncertain environments is presented in this paper. The design key factors for the navigation system of a...A navigation method based on the partially observable markov decision process （POMDP） for smart wheelchairs in uncertain environments is presented in this paper. The design key factors for the navigation system of a smart wheelchair are discussed. A kinematics model of the smart wheelchair is given, and the model and principle of POMDP are introduced. In order to respond in uncertain local environments, a novel navigation methodology based on POMDP using the sensors perception and the user＇s joystick input is presented. The state space, the action set, the observations and the sensor fusion of the navigation method are given in detail, and the optimal policy of the POMDP model is proposed. Experimental results demonstrate the feasibility of this navigation method. Analysis is also conducted to investigate performance evaluation, advantages of the approach and potential generalization of this paper.展开更多
Wireless body area networks （WBANs） can provide low-cost, timely healthcare services and are expected to be widely used for e-healthcare in hospitals. In a hospital, space is often limited and multiple WBANs have to...Wireless body area networks （WBANs） can provide low-cost, timely healthcare services and are expected to be widely used for e-healthcare in hospitals. In a hospital, space is often limited and multiple WBANs have to coexist in an area and share the same channel in order to provide healthcare services to different patients. This causes severe interference between WBANs that could significantly reduce the network throughput and increase the amount of power consumed by sensors placed on the body. There-fore, an efficient channel-resource allocation scheme in the medium access control （MAC） layer is crucial. In this paper, we devel-op a centralized MAC layer resource allocation scheme for a WBAN. We focus on mitigating the interference between WBANs and reducing the power consumed by sensors. Channel and buffer state are reported by smartphones deployed in each WBAN, and channel access allocation is performed by a central controller to maximize network throughput. Sensors have strict limitations in terms of energy consumption and computing capability and cannot provide all the necessary information for channel allocation in a timely manner. This deteriorates network performance. We exploit the temporal correlation of the body area channel in order to minimize the number of channel state reports necessary. We view the network design as a partly observable optimization prob-lem and develop a myopic policy, which we then simulate in Matlab.展开更多
I consider a system whose deterioration follows a discrete-time and discrete-state Markov chain with an absorbing state. When the system is put into practice, I may select operation (wait), imperfect repair, or replac...I consider a system whose deterioration follows a discrete-time and discrete-state Markov chain with an absorbing state. When the system is put into practice, I may select operation (wait), imperfect repair, or replacement at each discrete-time point. The true state of the system is not known when it is operated. Instead, the system is monitored after operation and some incomplete information concerned with the deterioration is obtained for decision making. Since there are multiple imperfect repairs, I can select one option from them when the imperfect repair is preferable to operation and replacement. To express this situation, I propose a POMDP model and theoretically investigate the structure of an optimal maintenance policy minimizing a total expected discounted cost for an unbounded horizon. Then two stochastic orders are used for the analysis of our problem.展开更多
基金supported by the Natural Science Foundation of China(NSFC)under Grant 62101051the Guangdong Key Laboratory of Intelligent Information ProcessingShenzhen Key Laboratory of Media Security,Shenzhen 518060,China。
文摘Capable of flexibly supporting diverse applications and providing computation services,the Mobile Edge Computing(MEC)-assisted Unmanned Aerial Vehicle(UAV)network is emerging as an innovational paradigm.In this paradigm,the heterogeneous resources of the network,including computing and communication resources,should be allocated properly to reduce computation and communication latency as well as energy consumption.However,most existing works solely focus on the optimization issues with global information,which is generally difficult to obtain in real-world scenarios.In this paper,fully considering the incomplete information resulting from diverse types of tasks,we study the joint task offloading and spectrum allocation problem in UAV network,where free UAV nodes serve as helpers for cooperative computation.The objective is to jointly optimize offloading mode,collaboration pairing,and channel allocation to minimize the weighted network cost.To achieve the purpose with only partial observation,an extensive-form game is introduced to reformulate the problem,and a regret learning-based scheme is proposed to achieve the equilibrium solution.With retrospective improvement property and information set concept,the designed algorithm is capable of combating incomplete information and obtaining more precise allocation patterns for diverse tasks.Numerical results show that our proposed algorithm outperforms the benchmarks across various settings.
基金supported by the National Key Research and Development Program of China(2022YFA1006103,2023YFA1009203)the National Natural Science Foundation of China(61925306,61821004,11831010,61977043,12001320)+2 种基金the Natural Science Foundation of Shandong Province(ZR2019ZD42,ZR2020ZD24)the Taishan Scholars Young Program of Shandong(TSQN202211032)the Young Scholars Program of Shandong University。
文摘This paper considers a linear-quadratic(LQ) meanfield game governed by a forward-backward stochastic system with partial observation and common noise,where a coupling structure enters state equations,cost functionals and observation equations.Firstly,to reduce the complexity of solving the meanfield game,a limiting control problem is introduced.By virtue of the decomposition approach,an admissible control set is proposed.Applying a filter technique and dimensional-expansion technique,a decentralized control strategy and a consistency condition system are derived,and the related solvability is also addressed.Secondly,we discuss an approximate Nash equilibrium property of the decentralized control strategy.Finally,we work out a financial problem with some numerical simulations.
基金supported by the NSF(No.DMS-1813071)(Chou)and the AFSOR(No.FA9550-22-1-0011)(Xiu).
文摘We present a numerical approach for modeling unknown dynamical systems using partially observed data,with a focus on biological systems with(relatively)complex dynamical behavior.As an extension of the recently developed deep neural network(DNN)learning methods,our approach is particularly suitable for practical situations when(i)measurement data are available for only a subset of the state variables,and(ii)the system parameters cannot be observed or measured at all.We demonstrate that,with a properly designed DNN structure with memory terms,effective DNN models can be learned from such partially observed data containing hidden parameters.The learned DNN model serves as an accurate predictive tool for system analysis.Through a few representative biological problems,we demonstrate that such DNN models can capture qualitative dynamical behavior changes in the system,such as bifurcations,even when the parameters controlling such behavior changes are completely unknown throughout not only the model learning process but also the system prediction process.The learned DNN model effectively creates a“closed”model involving only the observables when such a closed-form model does not exist mathematically.
基金The National Natural Science Foundation of China(No.60574006,60804017)
文摘To cope with multi-object tracking under real-world complex situations, a new video-based method is proposed. In the detecting step, the moving objects are segmented with the third level DWT （discrete wavelet transform ）and background difference. In the tracking step, the Kalman filter and scale parameter are used first to estimate the object position and bounding box. Then, the center-association-based projection ratio and region-association-based occlusion ratio are defined and combined to judge object behaviours. Finally, the tracking scheme and Kalman parameters are adaptively adjusted according to object behaviour. Under occlusion, partial observability is utilized to obtain the object measurements and optimum box dimensions. This method is robust in tracking mobile objects under such situations as occlusion, new appearing and stablization, etc. Experimental results show that the proposed method is efficient.
基金supported by the Aeronautical Science Foundation of China(20220013053005)。
文摘In this paper,a deep deterministic policy gradient algorithm based on Partially Observable Weighted Mean Field Reinforcement Learning(PO-WMFRL)framework is designed to solve the problem of path planning in large-scale Unmanned Aerial Vehicle(UAV)swarm operations.We establish a motion control and detection communication model of UAVs.A simulation environment is carried out with No-Fly Zone(NFZ),the task assembly point is established,and the long-term reward and immediate reward functions are designed for large-scale UAV swarm path planning problem.Considering the combat characteristics of large-scale UAV swarm,we improve the traditional Deep Deterministic Policy Gradient(DDPG)algorithm and propose a Partially Observable Weighted Mean Field Deep Deterministic Policy Gradient(PO-WMFDDPG)algorithm.The effectiveness of the PO-WMFDDPG algorithm is verified through simulation,and through the comparative analysis with the DDPG and MFDDPG algorithms,it is verified that the PO-WMFDDPG algorithm has a higher task success rate and convergence speed.
基金Project supported by the National Natural Science Foundation ofChina (No. 10332030), the Special Fund for Doctor Programs inInstitutions of Higher Learning of China (No. 20020335092), andthe Zhejiang Provincial Natural Science Foundation (No. 101046),China
文摘A stochastic optimal control strategy for partially observable nonlinear quasi Hamiltonian systems is proposed. The optimal control forces consist of two parts. The first part is determined by the conditions under which the stochastic optimal control problem of a partially observable nonlinear system is converted into that of a completely observable linear system. The second part is determined by solving the dynamical programming equation derived by applying the stochastic averaging method and stochastic dynamical programming principle to the completely observable linear control system. The response of the optimally controlled quasi Hamiltonian system is predicted by solving the averaged Fokker-Planck-Kolmogorov equation associated with the optimally controlled completely observable linear system and solving the Riccati equation for the estimated error of system states. An example is given to illustrate the procedure and effectiveness of the proposed control strategy.
基金supported by the National Natural Science Foundation of China(61473144)
文摘This paper investigates the fault detection problem for discrete event systems (DESs) which can be modeled by partially observed Petri nets (POPNs). To overcome the problem of low diagnosability in the POPN online fault diagnoser in current use, an improved online fault diagnosis algorithm that integrates generalized mutual exclusion constraints (GMECs) and integer linear programming (ILP) is proposed. Assume that the POPN structure and its initial markings are known, and the faults are modeled as unobservable transitions. First, the event sequence is observed and recorded. GMEC is used for elementary diagnosis of the system behavior, then the ILP problem of POPN is solved for further diagnosis. Finally, an example of a real DES to test the new fault diagnoser is analyzed. The proposed algorithm increases the diagnosability of the DES remarkably, and the effectiveness of the new algorithm integrating GMEC and ILP is verified.
基金supported by the National Natural Science Foundation of China (Nos. 10332030 and 10772159)Research Fund for doctoral Program of Higher Education of China (No. 20060335125)
文摘An optimal vibration control strategy for partially observable nonlinear quasi Hamiltonian systems with actuator saturation is proposed. First,a controlled partially observable non-linear system is converted into a completely observable linear control system of finite dimension based on the theorem due to Charalambous and Elliott. Then the partially averaged It stochastic differential equations and dynamical programming equation associated with the completely observable linear system are derived by using the stochastic averaging method and stochastic dynamical programming principle,respectively. The optimal control law is obtained from solving the final dynamical programming equation. The results show that the proposed control strategy has high control effectiveness and control effciency.
文摘The authors investigate the problem of impulse control of a partially observed diffusion process. The authors study the impulse control of Zakai type equations. The associated value function is characterized as the only viscosity solution of the corresponding quasi-variational inequality. The authors show the optimal cost function for the problem with incomplete information can be approximated by a sequence of value functions of the previous type.
基金supported by the National Natural Science Foundation of China(No.11861025)Guizhou QKZYD[2022](No.4055)Natural Science Research Project of Guizhou Provincial Department of Education(No.QJJ[2023]011)。
文摘A general model of insider trading on a dynamic asset in a finite time interval is proposed,in which an insider possesses the whole information on the dynamic values,noise traders without any information submit orders randomly as a martingale with volatility following a stochastic process,and market makers observe partial information when setting price in a semi-strong efficient way.With the help of filtering theory,BSDE method and dynamic programming principle,we establish a market equilibrium consisting of linear insider trading strategy and linear pricing rule,with the later characterized by price pressure on market orders and price pressure on asset observations.It shows that in the equilibrium,all the information on the risky asset is incorporated into the market price at the end of the transaction,and price pressure on market orders is a submartingale while market depth process is a martingale.Furthermore,as market makers'information precision on the asset tends to zero,the equilibrium with partial observation of market makers on the risky asset converges to the one without partial observation of market makers,while when market makers observe almost all of information on the asset,the expected profit earned by the insider makes almost zero,which is in accord with our economic intuition.Our results cover some classical results about continuous-time insider trading on a static asset.
基金supported by National Natural Science Foundation of China (Nos. 61174124, 61233003 and 60935001)National High Technology Research and Development Program of China (863 Program) (No. 2011AA01A102)
文摘In network service systems, satisfying quality of service (QoS) is one of the main objectives. Admission control and resource allocation strategy can be used to guarantee the QoS requirement. Based on partially observable Markov decision processes (POMDPs), this paper proposes a novel admission control model for video on demand (VOD) service systems with elastic QoS. Elastic QoS is also considered in resource allocation strategy. Policy gradient algorithm is often available to find the solution of POMDP problems, with a satisfactory convergence rate. Through numerical examples, it can be shown that the proposed admission control strategy has better performance than complete admission control strategy.
基金supported by the National Natural Science Foundation of China (51175502)
文摘Testing is the premise and foundation of realizing equipment health management （EHM）. To address the problem that the static periodic test strategy may cause deficient test or excessive test, a dynamic sequential test strategy （DSTS） for EHM is presented. Considering the situation that equipment health state is not completely observable in reality, a DSTS optimization method based on partially observable semi-Markov decision pro- cess （POSMDP） is proposed. Firstly, an equipment health state degradation model is constructed by Markov process, and the control limit maintenance policy is also introduced. Secondly, POSMDP is formulated in great detail. And then, POSMDP is converted to completely observable belief semi-Markov decision process （BSMDP） through belief state. The optimal equation and the corresponding optimal DSTS, which minimize the long-run ex- pected average cost per unit time, are obtained with BSMDP. The results of application in complex equipment show that the proposed DSTS is feasible and effective.
基金the National Natural Science Foundation of China(Grant Nos.61673027 and 62106047)the Beijing Social Science Foundation(Grant No.21GLC042)the Humanity and Social Science Youth foundation of Ministry of Education,China(Grant No.20YJCZH228)。
文摘This paper examines the effect of the observation time on source identification of a discrete-time susceptible-infectedrecovered diffusion process in a network with snapshot of partial nodes.We formulate the source identification problem as a maximum likelihood(ML)estimator and develop a statistical inference method based on Monte Carlo simulation(MCS)to estimate the source location and the initial time of diffusion.Experimental results in synthetic networks and real-world networks demonstrate evident impact of the observation time as well as the fraction of the observers on the concerned problem.
基金supported by the National Science Foundation for Young Scholars of China(6130123471401175)
文摘Multiple earth observing satellites need to communicate with each other to observe plenty of targets on the Earth together. The factors, such as external interference, result in satellite information interaction delays, which is unable to ensure the integrity and timeliness of the information on decision making for satellites. And the optimization of the planning result is affected. Therefore, the effect of communication delay is considered during the multi-satel ite coordinating process. For this problem, firstly, a distributed cooperative optimization problem for multiple satellites in the delayed communication environment is formulized. Secondly, based on both the analysis of the temporal sequence of tasks in a single satellite and the dynamically decoupled characteristics of the multi-satellite system, the environment information of multi-satellite distributed cooperative optimization is constructed on the basis of the directed acyclic graph（DAG）. Then, both a cooperative optimization decision making framework and a model are built according to the decentralized partial observable Markov decision process（DEC-POMDP）. After that, a satellite coordinating strategy aimed at different conditions of communication delay is mainly analyzed, and a unified processing strategy on communication delay is designed. An approximate cooperative optimization algorithm based on simulated annealing is proposed. Finally, the effectiveness and robustness of the method presented in this paper are verified via the simulation.
基金supported by the National Natural Science Foundation of China under Grant No.61301101
文摘The transmission delay of realtime video packet mainly depends on the sensing time delay(short-term factor) and the entire frame transmission delay(long-term factor).Therefore,the optimization problem in the spectrum handoff process should be formulated as the combination of microscopic optimization and macroscopic optimization.In this paper,we focus on the issue of combining these two optimization models,and propose a novel Evolution Spectrum Handoff(ESH)strategy to minimize the expected transmission delay of real-time video packet.In the microoptimized model,considering the tradeoff between Primary User's(PU's) allowable collision percentage of each channel and transmission delay of video packet,we propose a mixed integer non-linear programming scheme.The scheme is able to achieve the minimum sensing time which is termed as an optimal stopping time.In the macro-optimized model,using the optimal stopping time as reward function within the partially observable Markov decision process framework,the EHS strategy is designed to search an optimal target channel set and minimize the expected delay of packet in the long-term real-time video transmission.Meanwhile,the minimum expected transmission delay is obtained under practical cognitive radio networks' conditions,i.e.,secondary user's mobility,PU's random access,imperfect sensing information,etc..Theoretical analysis and simulation results show that the ESH strategy can effectively reduce the transmission delay of video packet in spectrum handoff process.
基金Project supported by the National Natural Science Foundation of China under a key grant (No.10332030)the Research Fund for the Doctoral Program of Higher Education of China (No.20060335125)the Zhejiang Provincial Natural Science Foundation of China (No.Y607087).
文摘A feedback control optimization method of partially observable linear structures via stationary response is proposed and analyzed with linear building structures equipped with control devices and sensors. First, the partially observable control problem of the structure under horizontal ground acceleration excitation is converted into a completely observable control problem. Then the It6 stochastic differential equations of the system are derived based on the stochastic averaging method for quasi-integrable Hamiltonian systems and the stationary solution to the Fokker-Plank-Kolmogorov （FPK） equation associated with the It6 equations is obtained. The performance index in terms of the mean system energy and mean square control force is established and the optimal control force is obtained by minimizing the performance index. Finally, the numerical results for a three-story building structure model under E1 Centro, Hachinohe, Northridge and Kobe earthquake excitations are given to illustrate the application and the effectiveness of the proposed method.
基金Corresponding au-thor:Ayman El Shenawy received the Ph.D.degree in systems and computer engineer-ing from Al-Azhar University,Egypt in 2013.He is currently working as a lecturer at Systems and Computers Engineering Department,Faculty of Engineering Al-Azhar University,Egypt.He already de-veloped some breakthrough research in the mentioned areas.He made significant con-tributions to the stated research fields.His research interests include artificial intelligent methods,robotics and machine learning.E-mail:eaymanelshenawy@azhar.edu.eg ORCID iD:0000-0002-1309-644。
文摘The multi-robot systems(MRS)exploration and fire searching problem is an important application of mobile robots which require massive computation capability that exceeds the ability of traditional MRS′s.This paper propose a cloud-based hybrid decentralized partially observable semi-Markov decision process(HDec-POSMDPs)model.The proposed model is implemented for MRS exploration and fire searching application based on the Internet of things(IoT)cloud robotics framework.In this implementation the heavy and expensive computational tasks are offloaded to the cloud servers.The proposed model achieves a significant improvement in the computation burden of the whole task relative to a traditional MRS.The proposed model is applied to explore and search for fire objects in an unknown environment;using different sets of robots sizes.The preliminary evaluation of this implementation demonstrates that as the parallelism of computational instances increase the delay of new actuation commands which will be decreased,the mean time of task completion is decreased,the number of turns in the path from the start pose cells to the target cells is minimized and the energy consumption for each robot is reduced.
文摘A navigation method based on the partially observable markov decision process （POMDP） for smart wheelchairs in uncertain environments is presented in this paper. The design key factors for the navigation system of a smart wheelchair are discussed. A kinematics model of the smart wheelchair is given, and the model and principle of POMDP are introduced. In order to respond in uncertain local environments, a novel navigation methodology based on POMDP using the sensors perception and the user＇s joystick input is presented. The state space, the action set, the observations and the sensor fusion of the navigation method are given in detail, and the optimal policy of the POMDP model is proposed. Experimental results demonstrate the feasibility of this navigation method. Analysis is also conducted to investigate performance evaluation, advantages of the approach and potential generalization of this paper.
基金supported by a research grant from the Natural Science and Engineering Research Council(NSERC)under grant No.CRDPJ 419147-11Care In Motion Inc.,Canada
文摘Wireless body area networks （WBANs） can provide low-cost, timely healthcare services and are expected to be widely used for e-healthcare in hospitals. In a hospital, space is often limited and multiple WBANs have to coexist in an area and share the same channel in order to provide healthcare services to different patients. This causes severe interference between WBANs that could significantly reduce the network throughput and increase the amount of power consumed by sensors placed on the body. There-fore, an efficient channel-resource allocation scheme in the medium access control （MAC） layer is crucial. In this paper, we devel-op a centralized MAC layer resource allocation scheme for a WBAN. We focus on mitigating the interference between WBANs and reducing the power consumed by sensors. Channel and buffer state are reported by smartphones deployed in each WBAN, and channel access allocation is performed by a central controller to maximize network throughput. Sensors have strict limitations in terms of energy consumption and computing capability and cannot provide all the necessary information for channel allocation in a timely manner. This deteriorates network performance. We exploit the temporal correlation of the body area channel in order to minimize the number of channel state reports necessary. We view the network design as a partly observable optimization prob-lem and develop a myopic policy, which we then simulate in Matlab.
文摘I consider a system whose deterioration follows a discrete-time and discrete-state Markov chain with an absorbing state. When the system is put into practice, I may select operation (wait), imperfect repair, or replacement at each discrete-time point. The true state of the system is not known when it is operated. Instead, the system is monitored after operation and some incomplete information concerned with the deterioration is obtained for decision making. Since there are multiple imperfect repairs, I can select one option from them when the imperfect repair is preferable to operation and replacement. To express this situation, I propose a POMDP model and theoretically investigate the structure of an optimal maintenance policy minimizing a total expected discounted cost for an unbounded horizon. Then two stochastic orders are used for the analysis of our problem.
