This paper addresses an indefinite linear-quadratic mean field games for stochastic largepopulation system,where the individual diffusion coefficients may depend on both the state and the control of the agents.Moreove...This paper addresses an indefinite linear-quadratic mean field games for stochastic largepopulation system,where the individual diffusion coefficients may depend on both the state and the control of the agents.Moreover,the control weights in the cost functionals could be indefinite.The authors employ a direct approach to derive theε-Nash equilibrium strategy.First,the authors formally solve an N-player game problem within a vast but finite population setting.Subsequently,by introducing two Riccati equations,the authors decouple or reduce the high-dimensional systems to yield centralized strategies,which depend on the state of a specific player and the average state of the population.As the population size N goes infinity,the construction of decentralized strategies becomes feasible.Then,the authors demonstrate that these strategies constitute anε-Nash equilibrium.Finally,numerical examples are provided to demonstrate the effectiveness of the proposed strategies.展开更多
This paper studies a class of linear quadratic mean field games where the coefficients of quadratic cost functions depend on both the mean and the variance of the population’s state distribution through its quantile ...This paper studies a class of linear quadratic mean field games where the coefficients of quadratic cost functions depend on both the mean and the variance of the population’s state distribution through its quantile function.Such a formulation allows for modelling agents that are sensitive to not only the population average but also the population variance.The potential mean field game equilibria are identified.Their calculation involves solving two nonlinearly coupled differential equations:one is a Riccati equation and the other the variance evolution equation.Sufficient conditions for the existence and uniqueness of a mean field equilibrium are established.Finally,numerical results are presented to illustrate the behavior of two coupled differential equations and the performance of the mean field game solution.展开更多
This paper studies an asymptotic solvability problem for linear quadratic(LQ)mean field games with controlled diffusions and indefinite weights for the state and control in the costs.The authors employ a rescaling app...This paper studies an asymptotic solvability problem for linear quadratic(LQ)mean field games with controlled diffusions and indefinite weights for the state and control in the costs.The authors employ a rescaling approach to derive a low dimensional Riccati ordinary differential equation(ODE)system,which characterizes a necessary and sufficient condition for asymptotic solvability.The rescaling technique is further used for performance estimates,establishing an O(1/N)-Nash equilibrium for the obtained decentralized strategies.展开更多
In this paper,the authors consider the mean field game with a common noise and allow the state coefficients to vary with the conditional distribution in a nonlinear way.They assume that the cost function satisfies a c...In this paper,the authors consider the mean field game with a common noise and allow the state coefficients to vary with the conditional distribution in a nonlinear way.They assume that the cost function satisfies a convexity and a weak monotonicity property.They use the sufficient Pontryagin principle for optimality to transform the mean field control problem into existence and uniqueness of solution of conditional distribution dependent forward-backward stochastic differential equation(FBSDE for short).They prove the existence and uniqueness of solution of the conditional distribution dependent FBSDE when the dependence of the state on the conditional distribution is sufficiently small,or when the convexity parameter of the running cost on the control is sufficiently large.Two different methods are developed.The first method is based on a continuation of the coefficients,which is developed for FBSDE by[Hu,Y.and Peng,S.,Solution of forward-backward stochastic differential equations,Probab.Theory Rel.,103(2),1995,273–283].They apply the method to conditional distribution dependent FBSDE.The second method is to show the existence result on a small time interval by Banach fixed point theorem and then extend the local solution to the whole time interval.展开更多
In this paper,a deep deterministic policy gradient algorithm based on Partially Observable Weighted Mean Field Reinforcement Learning(PO-WMFRL)framework is designed to solve the problem of path planning in large-scale...In this paper,a deep deterministic policy gradient algorithm based on Partially Observable Weighted Mean Field Reinforcement Learning(PO-WMFRL)framework is designed to solve the problem of path planning in large-scale Unmanned Aerial Vehicle(UAV)swarm operations.We establish a motion control and detection communication model of UAVs.A simulation environment is carried out with No-Fly Zone(NFZ),the task assembly point is established,and the long-term reward and immediate reward functions are designed for large-scale UAV swarm path planning problem.Considering the combat characteristics of large-scale UAV swarm,we improve the traditional Deep Deterministic Policy Gradient(DDPG)algorithm and propose a Partially Observable Weighted Mean Field Deep Deterministic Policy Gradient(PO-WMFDDPG)algorithm.The effectiveness of the PO-WMFDDPG algorithm is verified through simulation,and through the comparative analysis with the DDPG and MFDDPG algorithms,it is verified that the PO-WMFDDPG algorithm has a higher task success rate and convergence speed.展开更多
The theory of Mean Field Game of Controls considers a class of mean field games where the interaction is through the joint distribution of the state and control.It is well known that,for standard mean field games,cert...The theory of Mean Field Game of Controls considers a class of mean field games where the interaction is through the joint distribution of the state and control.It is well known that,for standard mean field games,certain monotonicity conditions are crucial to guarantee the uniqueness of mean field equilibria and then the global wellposedness for master equations.In the literature the monotonicity condition could be the Lasry–Lions monotonicity,the displacement monotonicity,or the anti-monotonicity conditions.In this paper,we investigate these three types of monotonicity conditions for Mean Field Games of Controls and show their propagation along the solutions to the master equations with common noises.In particular,we extend the displacement monotonicity to semi-monotonicity,whose propagation result is new even for standard mean field games.This is the first step towards the global wellposedness theory for master equations of Mean Field Games of Controls.展开更多
With energy harvesting capability, the Internet of things(IoT) devices transmit data depending on their available energy, which leads to a more complicated coupling and brings new technical challenges to delay optimiz...With energy harvesting capability, the Internet of things(IoT) devices transmit data depending on their available energy, which leads to a more complicated coupling and brings new technical challenges to delay optimization. In this paper,we study the delay-optimal random access(RA) in large-scale energy harvesting IoT networks. We model a two-dimensional Markov decision process(MDP)to address the coupling between the data and energy queues, and adopt the mean field game(MFG) theory to reveal the coupling among the devices by utilizing the large-scale property. Specifically, to obtain the optimal access strategy for each device, we derive the Hamilton-Jacobi-Bellman(HJB) equation which requires the statistical information of other devices.Moreover, to model the evolution of the states distribution in the system, we derive the Fokker-PlanckKolmogorov(FPK) equation based on the access strategy of devices. By solving the two coupled equations,we obtain the delay-optimal random access solution in an iterative manner with Lax-Friedrichs method. Finally, the simulation results show that the proposed scheme achieves significant performance gain compared with the conventional schemes.展开更多
With the development of the Internet of Things,the edge devices are increasing.Cyber security issues in edge computing have also emerged and caused great concern.We propose a defense strategy based on Mean field game ...With the development of the Internet of Things,the edge devices are increasing.Cyber security issues in edge computing have also emerged and caused great concern.We propose a defense strategy based on Mean field game to solve the security issues of edge user data during edge computing.Firstly,an individual cost function is formulated to build an edge user data security defense model.Secondly,we research the𝜀𝜀-Nash equilibrium of the individual cost function with finite players and prove the existence of the optimal defense strategy.Finally,by analyzing the stability of edge user data loss,it proves that the proposed defense strategy is effective.展开更多
In the 6G Internet of Things(IoT)paradigm,unprecedented challenges will be raised to provide massive connectivity,ultra-low latency,and energy efficiency for ultra-dense IoT devices.To address these challenges,we expl...In the 6G Internet of Things(IoT)paradigm,unprecedented challenges will be raised to provide massive connectivity,ultra-low latency,and energy efficiency for ultra-dense IoT devices.To address these challenges,we explore the non-orthogonal multiple access(NOMA)based grant-free random access(GFRA)schemes in the cellular uplink to support massive IoT devices with high spectrum efficiency and low access latency.In particular,we focus on optimizing the backoff strategy of each device when transmitting time-sensitive data samples to a multiple-input multiple-output(MIMO)-enabled base station subject to energy constraints.To cope with the dynamic varied channel and the severe uplink interference due to the uncoordinated grant-free access,we formulate the optimization problem as a multi-user non-cooperative dynamic stochastic game(MUN-DSG).To avoid dimensional disaster as the device number grows large,the optimization problem is transformed into a mean field game(MFG),and its Nash equilibrium can be achieved by solving the corresponding Hamilton-Jacobi-Bellman(HJB)and Fokker-Planck-Kolmogorov(FPK)equations.Thus,a Mean Field-based Dynamic Backoff(MFDB)scheme is proposed as the optimal GFRA solution for each device.Extensive simulation has been fulfilled to compare the proposed MFDB with contemporary random access approaches like access class barring(ACB),slotted-Additive LinksOn-lineHawaii Area(ALOHA),andminimum backoff(MB)under both static and dynamic channels,and the results proved thatMFDB can achieve the least access delay and cumulated cost during multiple transmission frames.展开更多
We study n-player games of portfolio choice in general common Ito-diffusion markets under relative performance criteria and time monotone forward utilities.We,also,consider their continuum limit which gives rise to a ...We study n-player games of portfolio choice in general common Ito-diffusion markets under relative performance criteria and time monotone forward utilities.We,also,consider their continuum limit which gives rise to a forward mean field game with unbounded controls in both the drift and volatility terms.Furthermore,we allow for general(time monotone)preferences,thus departing from the homothetic case,the only case so far analyzed.We produce explicit solutions for the optimal policies,the optimal wealth processes and the game values,and also provide representative examples for both the finite and the mean field game.展开更多
This paper is concerned with the linear-quadratic social optima for a class of N weakly coupled backward system with partial information structure.The system dynamics are governed by linear backward stochastic differe...This paper is concerned with the linear-quadratic social optima for a class of N weakly coupled backward system with partial information structure.The system dynamics are governed by linear backward stochastic differential equations,and the objective is to minimize a social cost.The stochastic filtering Hamiltonian system is obtained from variational analysis.By virtue of the stochastic filtering technique and backward decoupling method,the feedback form of optimal control is derived.Aiming to overcome the curse of dimensionality and reduce the information requirements,we design a set of decentralized control laws,which is further shown to be asymptotic.Finally,an example of the scalar-valued case is studied.展开更多
In this paper,we study the n-player game and the mean field game under the constant relative risk aversion relative performance on terminal wealth,in which the interaction occurs by peer competition.In the model with ...In this paper,we study the n-player game and the mean field game under the constant relative risk aversion relative performance on terminal wealth,in which the interaction occurs by peer competition.In the model with n agents,the price dynamics of underlying risky assets depend on a common noise and contagious jump risk modeled by a multi-dimensional nonlinear Hawkes process.With a continuum of agents,we formulate the mean field game problem and characterize a deterministic mean field equilibrium in an analytical form under some conditions,allowing us to investigate some impacts of model parameters in the limiting model and discuss some financial implications.Moreover,based on the mean field equilibrium,we construct an approximate Nash equilibrium for the n-player game when n is sufficiently large.The explicit order of the approximation error is also derived.展开更多
We introduce a class of systems of Hamilton-Jacobi equations characterizing geodesic centroidal tessellations,i.e.,tessellations of domains with respect to geodesic distances where generators and centroids coincide.Ty...We introduce a class of systems of Hamilton-Jacobi equations characterizing geodesic centroidal tessellations,i.e.,tessellations of domains with respect to geodesic distances where generators and centroids coincide.Typical examples are given by geodesic centroi-dal Voronoi tessellations and geodesic centroidal power diagrams.An appropriate version of the Fast Marching method on unstructured grids allows computing the solution of the Hamilton-Jacobi system and,therefore,the associated tessellations.We propose various numerical examples to illustrate the features of the technique.展开更多
We consider an optimal control problem which serves as a mathematical model for several problems in economics and management.The problem is the minimization of a continuous constrained functional governed by a linear ...We consider an optimal control problem which serves as a mathematical model for several problems in economics and management.The problem is the minimization of a continuous constrained functional governed by a linear parabolic diffusion-advection equation controlled in a coefficient in advection part.The additional constraint is non-negativity of a solution of state equation.We construct and analyze several mesh schemes approximating the formulated problem using finite difference methods in space and in time.All these approximations keep the positivity of the solutions to mesh state problem,either unconditionally or under some additional constraints to mesh steps.This allows us to remove corresponding constraint from the formulation of the discrete problem to simplify its implementation.Based on theoretical estimates and numerical results,we draw conclusions about the quality of the proposed mesh schemes.展开更多
基金supported by National Key R&D Program of China under Grant No.2022YFA1006104the National Natural Science Foundations of China under Grant Nos.12471419 and 12271304Shandong Provincial Natural Science Foundations under Grant Nos.ZR2024ZD35 and ZR2022JQ01。
文摘This paper addresses an indefinite linear-quadratic mean field games for stochastic largepopulation system,where the individual diffusion coefficients may depend on both the state and the control of the agents.Moreover,the control weights in the cost functionals could be indefinite.The authors employ a direct approach to derive theε-Nash equilibrium strategy.First,the authors formally solve an N-player game problem within a vast but finite population setting.Subsequently,by introducing two Riccati equations,the authors decouple or reduce the high-dimensional systems to yield centralized strategies,which depend on the state of a specific player and the average state of the population.As the population size N goes infinity,the construction of decentralized strategies becomes feasible.Then,the authors demonstrate that these strategies constitute anε-Nash equilibrium.Finally,numerical examples are provided to demonstrate the effectiveness of the proposed strategies.
基金supported by NSERC(Canada)under Grant Nos.RGPIN-2024-06612(SG)and RGPIN 2022-05402(RM).
文摘This paper studies a class of linear quadratic mean field games where the coefficients of quadratic cost functions depend on both the mean and the variance of the population’s state distribution through its quantile function.Such a formulation allows for modelling agents that are sensitive to not only the population average but also the population variance.The potential mean field game equilibria are identified.Their calculation involves solving two nonlinearly coupled differential equations:one is a Riccati equation and the other the variance evolution equation.Sufficient conditions for the existence and uniqueness of a mean field equilibrium are established.Finally,numerical results are presented to illustrate the behavior of two coupled differential equations and the performance of the mean field game solution.
基金Natural Sciences and Engineering Research Council(NSERC)of Canada。
文摘This paper studies an asymptotic solvability problem for linear quadratic(LQ)mean field games with controlled diffusions and indefinite weights for the state and control in the costs.The authors employ a rescaling approach to derive a low dimensional Riccati ordinary differential equation(ODE)system,which characterizes a necessary and sufficient condition for asymptotic solvability.The rescaling technique is further used for performance estimates,establishing an O(1/N)-Nash equilibrium for the obtained decentralized strategies.
基金supported by the National Natural Science Foundation of China(No.12031009)。
文摘In this paper,the authors consider the mean field game with a common noise and allow the state coefficients to vary with the conditional distribution in a nonlinear way.They assume that the cost function satisfies a convexity and a weak monotonicity property.They use the sufficient Pontryagin principle for optimality to transform the mean field control problem into existence and uniqueness of solution of conditional distribution dependent forward-backward stochastic differential equation(FBSDE for short).They prove the existence and uniqueness of solution of the conditional distribution dependent FBSDE when the dependence of the state on the conditional distribution is sufficiently small,or when the convexity parameter of the running cost on the control is sufficiently large.Two different methods are developed.The first method is based on a continuation of the coefficients,which is developed for FBSDE by[Hu,Y.and Peng,S.,Solution of forward-backward stochastic differential equations,Probab.Theory Rel.,103(2),1995,273–283].They apply the method to conditional distribution dependent FBSDE.The second method is to show the existence result on a small time interval by Banach fixed point theorem and then extend the local solution to the whole time interval.
基金supported by the Aeronautical Science Foundation of China(20220013053005)。
文摘In this paper,a deep deterministic policy gradient algorithm based on Partially Observable Weighted Mean Field Reinforcement Learning(PO-WMFRL)framework is designed to solve the problem of path planning in large-scale Unmanned Aerial Vehicle(UAV)swarm operations.We establish a motion control and detection communication model of UAVs.A simulation environment is carried out with No-Fly Zone(NFZ),the task assembly point is established,and the long-term reward and immediate reward functions are designed for large-scale UAV swarm path planning problem.Considering the combat characteristics of large-scale UAV swarm,we improve the traditional Deep Deterministic Policy Gradient(DDPG)algorithm and propose a Partially Observable Weighted Mean Field Deep Deterministic Policy Gradient(PO-WMFDDPG)algorithm.The effectiveness of the PO-WMFDDPG algorithm is verified through simulation,and through the comparative analysis with the DDPG and MFDDPG algorithms,it is verified that the PO-WMFDDPG algorithm has a higher task success rate and convergence speed.
基金Chenchen Mou is supported in part by CityU Start-up(Grant No.7200684)Hong Kong RGC(Grant No.ECS 9048215).Jianfeng Zhang is supported in part by NSF(Grant Nos.DMS-1908665 and DMS-2205972).
文摘The theory of Mean Field Game of Controls considers a class of mean field games where the interaction is through the joint distribution of the state and control.It is well known that,for standard mean field games,certain monotonicity conditions are crucial to guarantee the uniqueness of mean field equilibria and then the global wellposedness for master equations.In the literature the monotonicity condition could be the Lasry–Lions monotonicity,the displacement monotonicity,or the anti-monotonicity conditions.In this paper,we investigate these three types of monotonicity conditions for Mean Field Games of Controls and show their propagation along the solutions to the master equations with common noises.In particular,we extend the displacement monotonicity to semi-monotonicity,whose propagation result is new even for standard mean field games.This is the first step towards the global wellposedness theory for master equations of Mean Field Games of Controls.
基金supported in part by Key R&D Program of Zhejiang (No. 2022C03078)National Natural Science Foundation of China (No. U20A20158)+1 种基金National Key R&D Program of China (No. 2018YFB1801104)Ningbo S&T Major Project (No. 2019B10079)。
文摘With energy harvesting capability, the Internet of things(IoT) devices transmit data depending on their available energy, which leads to a more complicated coupling and brings new technical challenges to delay optimization. In this paper,we study the delay-optimal random access(RA) in large-scale energy harvesting IoT networks. We model a two-dimensional Markov decision process(MDP)to address the coupling between the data and energy queues, and adopt the mean field game(MFG) theory to reveal the coupling among the devices by utilizing the large-scale property. Specifically, to obtain the optimal access strategy for each device, we derive the Hamilton-Jacobi-Bellman(HJB) equation which requires the statistical information of other devices.Moreover, to model the evolution of the states distribution in the system, we derive the Fokker-PlanckKolmogorov(FPK) equation based on the access strategy of devices. By solving the two coupled equations,we obtain the delay-optimal random access solution in an iterative manner with Lax-Friedrichs method. Finally, the simulation results show that the proposed scheme achieves significant performance gain compared with the conventional schemes.
文摘With the development of the Internet of Things,the edge devices are increasing.Cyber security issues in edge computing have also emerged and caused great concern.We propose a defense strategy based on Mean field game to solve the security issues of edge user data during edge computing.Firstly,an individual cost function is formulated to build an edge user data security defense model.Secondly,we research the𝜀𝜀-Nash equilibrium of the individual cost function with finite players and prove the existence of the optimal defense strategy.Finally,by analyzing the stability of edge user data loss,it proves that the proposed defense strategy is effective.
基金supported by the National Natural Science Foundation of China underGrant 62371036,supported authors Haibo Wang,Hongwei Gao and Pai Jiang.
文摘In the 6G Internet of Things(IoT)paradigm,unprecedented challenges will be raised to provide massive connectivity,ultra-low latency,and energy efficiency for ultra-dense IoT devices.To address these challenges,we explore the non-orthogonal multiple access(NOMA)based grant-free random access(GFRA)schemes in the cellular uplink to support massive IoT devices with high spectrum efficiency and low access latency.In particular,we focus on optimizing the backoff strategy of each device when transmitting time-sensitive data samples to a multiple-input multiple-output(MIMO)-enabled base station subject to energy constraints.To cope with the dynamic varied channel and the severe uplink interference due to the uncoordinated grant-free access,we formulate the optimization problem as a multi-user non-cooperative dynamic stochastic game(MUN-DSG).To avoid dimensional disaster as the device number grows large,the optimization problem is transformed into a mean field game(MFG),and its Nash equilibrium can be achieved by solving the corresponding Hamilton-Jacobi-Bellman(HJB)and Fokker-Planck-Kolmogorov(FPK)equations.Thus,a Mean Field-based Dynamic Backoff(MFDB)scheme is proposed as the optimal GFRA solution for each device.Extensive simulation has been fulfilled to compare the proposed MFDB with contemporary random access approaches like access class barring(ACB),slotted-Additive LinksOn-lineHawaii Area(ALOHA),andminimum backoff(MB)under both static and dynamic channels,and the results proved thatMFDB can achieve the least access delay and cumulated cost during multiple transmission frames.
文摘We study n-player games of portfolio choice in general common Ito-diffusion markets under relative performance criteria and time monotone forward utilities.We,also,consider their continuum limit which gives rise to a forward mean field game with unbounded controls in both the drift and volatility terms.Furthermore,we allow for general(time monotone)preferences,thus departing from the homothetic case,the only case so far analyzed.We produce explicit solutions for the optimal policies,the optimal wealth processes and the game values,and also provide representative examples for both the finite and the mean field game.
基金supported jointly by the Natural Science Foundation of China[No.11831010]and[No.61961160732]the Natural Science Foundation of Shandong Province[No.ZR2019ZD42]the Taishan Scholars Climbing Program of Shandong[No.TSPD20210302].
文摘This paper is concerned with the linear-quadratic social optima for a class of N weakly coupled backward system with partial information structure.The system dynamics are governed by linear backward stochastic differential equations,and the objective is to minimize a social cost.The stochastic filtering Hamiltonian system is obtained from variational analysis.By virtue of the stochastic filtering technique and backward decoupling method,the feedback form of optimal control is derived.Aiming to overcome the curse of dimensionality and reduce the information requirements,we design a set of decentralized control laws,which is further shown to be asymptotic.Finally,an example of the scalar-valued case is studied.
基金supported by Natural Science Basic Research Program of Shaanxi(Grant No.2023-JC-JQ-05)National Natural Science Foundation of China(Grant No.11971368)+1 种基金supported by the Fundamental Research Funds for the Central Universities(Grant No.WK3470000024)supported by The Hong Kong Polytechnic University(Grant Nos.P0031417 and P0039251)。
文摘In this paper,we study the n-player game and the mean field game under the constant relative risk aversion relative performance on terminal wealth,in which the interaction occurs by peer competition.In the model with n agents,the price dynamics of underlying risky assets depend on a common noise and contagious jump risk modeled by a multi-dimensional nonlinear Hawkes process.With a continuum of agents,we formulate the mean field game problem and characterize a deterministic mean field equilibrium in an analytical form under some conditions,allowing us to investigate some impacts of model parameters in the limiting model and discuss some financial implications.Moreover,based on the mean field equilibrium,we construct an approximate Nash equilibrium for the n-player game when n is sufficiently large.The explicit order of the approximation error is also derived.
基金funding provided by Politecnico di Torino within the CRUI-CARE Agreement.The present research was partially supported by MIUR Grant“Dipartimenti Eccellenza 2018-2022”CUP:E11G18000350001DISMA,Politecnico di Torino and by the Italian Ministry for University and Research(MUR)through the PRIN 2020 project“Integrated Mathematical Approaches to Socio-Epidemiological Dynamics”(No.2020JLWP23,CUP:E15F21005420006).
文摘We introduce a class of systems of Hamilton-Jacobi equations characterizing geodesic centroidal tessellations,i.e.,tessellations of domains with respect to geodesic distances where generators and centroids coincide.Typical examples are given by geodesic centroi-dal Voronoi tessellations and geodesic centroidal power diagrams.An appropriate version of the Fast Marching method on unstructured grids allows computing the solution of the Hamilton-Jacobi system and,therefore,the associated tessellations.We propose various numerical examples to illustrate the features of the technique.
基金Shuhua Zhang was supported by the National Basic Research Program(No.2012CB955804)the Major Research Plan of the National Natural Science Foundation of China(No.91430108)+2 种基金the Natural Science Foundation of China(No.11771322)the Major Program of Tianjin University of Finance and Economics(No.ZD1302)Alexander Lapin was supported by Russian Foundation of Basic Researches(No.16-01-00408)and by program”1000 Talents”of China.
文摘We consider an optimal control problem which serves as a mathematical model for several problems in economics and management.The problem is the minimization of a continuous constrained functional governed by a linear parabolic diffusion-advection equation controlled in a coefficient in advection part.The additional constraint is non-negativity of a solution of state equation.We construct and analyze several mesh schemes approximating the formulated problem using finite difference methods in space and in time.All these approximations keep the positivity of the solutions to mesh state problem,either unconditionally or under some additional constraints to mesh steps.This allows us to remove corresponding constraint from the formulation of the discrete problem to simplify its implementation.Based on theoretical estimates and numerical results,we draw conclusions about the quality of the proposed mesh schemes.
