New types of aerial robots(NTARs)have found extensive applications in the military,civilian contexts,scientific research,disaster management,and various other domains.Compared with traditional aerial robots,NTARs exhi...New types of aerial robots(NTARs)have found extensive applications in the military,civilian contexts,scientific research,disaster management,and various other domains.Compared with traditional aerial robots,NTARs exhibit a broader range of morphological diversity,locomotion capabilities,and enhanced operational capacities.Therefore,this study defines aerial robots with the four characteristics of morphability,biomimicry,multi-modal locomotion,and manipulator attachment as NTARs.Subsequently,this paper discusses the latest research progress in the materials and manufacturing technology,actuation technology,and perception and control technology of NTARs.Thereafter,the research status of NTAR systems is summarized,focusing on the frontier development and application cases of flapping-wing microair vehicles,perching aerial robots,amphibious robots,and operational aerial robots.Finally,the main challenges presented by NTARs in terms of energy,materials,and perception are analyzed,and the future development trends of NTARs are summarized in terms of size and endurance,mechatronics,and complex scenarios,providing a reference direction for the follow-up exploration of NTARs.展开更多
This paper develops a novel hierarchical control strategy for improving the trajectory tracking capability of aerial robots under parameter uncertainties.The hierarchical control strategy is composed of an adaptive sl...This paper develops a novel hierarchical control strategy for improving the trajectory tracking capability of aerial robots under parameter uncertainties.The hierarchical control strategy is composed of an adaptive sliding mode controller and a model-free iterative sliding mode controller(MFISMC).A position controller is designed based on adaptive sliding mode control(SMC)to safely drive the aerial robot and ensure fast state convergence under external disturbances.Additionally,the MFISMC acts as an attitude controller to estimate the unmodeled dynamics without detailed knowledge of aerial robots.Then,the adaption laws are derived with the Lyapunov theory to guarantee the asymptotic tracking of the system state.Finally,to demonstrate the performance and robustness of the proposed control strategy,numerical simulations are carried out,which are also compared with other conventional strategies,such as proportional-integralderivative(PID),backstepping(BS),and SMC.The simulation results indicate that the proposed hierarchical control strategy can fulfill zero steady-state error and achieve faster convergence compared with conventional strategies.展开更多
Since precise self-position estimation is required for autonomous flight of aerial robots, there has been some studies on self-position estimation of indoor aerial robots. In this study, we tackle the self-position es...Since precise self-position estimation is required for autonomous flight of aerial robots, there has been some studies on self-position estimation of indoor aerial robots. In this study, we tackle the self-position estimation problem by mounting a small downward-facing camera on the chassis of an aerial robot. We obtain robot position by sensing the features on the indoor floor.In this work, we used the vertex points(tile corners) where four tiles on a typical tiled floor connected, as an existing feature of the floor. Furthermore, a small lightweight microcontroller is mounted on the robot to perform image processing for the onboard camera. A lightweight image processing algorithm is developed. So, the real-time image processing could be performed by the microcontroller alone which leads to conduct on-board real time tile corner detection. Furthermore, same microcontroller performs control value calculation for flight commanding. The flight commands are implemented based on the detected tile corner information. The above mentioned all devices are mounted on an actual machine, and the effectiveness of the system was investigated.展开更多
In aerial robots＇ visual navigation, it is essential yet very difficult to detect the attitude and position of the robots operated in real time. By introducing a new parametric model, the problem can be reduced from ...In aerial robots＇ visual navigation, it is essential yet very difficult to detect the attitude and position of the robots operated in real time. By introducing a new parametric model, the problem can be reduced from almost unmanageable to be partly solved, though not fully, as per the requirement. In this parametric approach, a multi-scale least square method is formulated first. By propagating as well as improving the parameters down from layer to layer of the image pyramid, a new global feature line can then be detected to parameterize the attitude of the robots. Furthermore, this approach paves the way for segmenting the image into distinct parts, which can be realized by deploying a Bayesian classifier on the picture cell level. Comparison with the Hough transform based method in terms of robustness and precision shows that this multi-scale least square algorithm is considerably more robust to noises. Some discussions are also given.展开更多
The intersection of Quantum Technologies and Robotics Autonomy is explored in the present paper.The two areas are brought together in establishing an interdisciplinary interface that contributes to advancing the field...The intersection of Quantum Technologies and Robotics Autonomy is explored in the present paper.The two areas are brought together in establishing an interdisciplinary interface that contributes to advancing the field of system autonomy,and pushes the engineering boundaries beyond the existing techniques.The present research adopts the experimental aspects of quantum entanglement and quantum cryptography,and integrates these established quantum capabilities into distributed robotic platforms,to explore the possibility of achieving increased autonomy for the control of multi-agent robotic systems engaged in cooperative tasks.Experimental quantum capabilities are realized by producing single photons(using spontaneous parametric down-conversion process),polarization of photons,detecting vertical and horizontal polarizations,and single photon detecting/counting.Specifically,such quantum aspects are implemented on network of classical agents,i.e.,classical aerial and ground robots/unmanned systems.With respect to classical systems for robotic applications,leveraging quantum technology is expected to lead to guaranteed security,very fast control and communication,and unparalleled quantum capabilities such as entanglement and quantum superposition that will enable novel applications.展开更多
This paper introduces a new algorithm for estimating the relative pose of a moving camera using consecutive frames of a video sequence. State-of-the-art algorithms for calculating the relative pose between two images ...This paper introduces a new algorithm for estimating the relative pose of a moving camera using consecutive frames of a video sequence. State-of-the-art algorithms for calculating the relative pose between two images use matching features to estimate the essential matrix. The essential matrix is then decomposed into the relative rotation and normalized translation between frames. To be robust to noise and feature match outliers, these methods generate a large number of essential matrix hypotheses from randomly selected minimal subsets of feature pairs, and then score these hypotheses on all feature pairs. Alternatively, the algorithm introduced in this paper calculates relative pose hypotheses by directly optimizing the rotation and normalized translation between frames, rather than calculating the essential matrix and then performing the decomposition. The resulting algorithm improves computation time by an order of magnitude. If an inertial measurement unit(IMU) is available, it is used to seed the optimizer, and in addition, we reuse the best hypothesis at each iteration to seed the optimizer thereby reducing the number of relative pose hypotheses that must be generated and scored. These advantages greatly speed up performance and enable the algorithm to run in real-time on low cost embedded hardware. We show application of our algorithm to visual multi-target tracking(MTT) in the presence of parallax and demonstrate its real-time performance on a 640 × 480 video sequence captured on a UAV. Video results are available at https://youtu.be/Hh K-p2 h XNn U.展开更多
Purpose–The purpose of this paper is to describe the specification language TML for adaptive mission plans that the authors designed and implemented for the open-source framework Aerostack for aerial robotics.Design/...Purpose–The purpose of this paper is to describe the specification language TML for adaptive mission plans that the authors designed and implemented for the open-source framework Aerostack for aerial robotics.Design/methodology/approach–The TML language combines a task-based hierarchical approach together with a more flexible representation,rule-based reactive planning,to facilitate adaptability.This approach includes additional notions that abstract programming details.The authors built an interpreter integrated in the software framework Aerostack.The interpreter was validated with flight experiments for multi-robot missions in dynamic environments.Findings–The experiments proved that the TML language is easy to use and expressive enough to formulate adaptive missions in dynamic environments.The experiments also showed that the TML interpreter is efficient to execute multi-robot aerial missions and reusable for different platforms.The TML interpreter is able to verify the mission plan before its execution,which increases robustness and safety,avoiding the execution of certain plans that are not feasible.Originality/value–One of the main contributions of this work is the availability of a reliable solution to specify aerial mission plans,integrated in an active open-source project with periodic releases.To the best knowledge of the authors,there are not solutions similar to this in other active open-source projects.As additional contributions,TML uses an original combination of representations for adaptive mission plans(i.e.task trees with original abstract notions and rule-based reactive planning)together with the demonstration of its adequacy for aerial robotics.展开更多
Drones have increasingly collaborated with human workers in some workspaces,such as warehouses.The failure of a drone flight may bring potential risks to human beings'life safety during some aerial tasks.One of th...Drones have increasingly collaborated with human workers in some workspaces,such as warehouses.The failure of a drone flight may bring potential risks to human beings'life safety during some aerial tasks.One of the most common flight failures is triggered by damaged propellers.To quickly detect physical damage to propellers,recognise risky flights,and provide early warnings to surrounding human workers,a new and compre-hensive fault diagnosis framework is presented that uses only the audio caused by pro-peller rotation without accessing any flight data.The diagnosis framework includes three components:leverage convolutional neural networks,transfer learning,and Bayesian optimisation.Particularly,the audio signal from an actual flight is collected and trans-ferred into time–frequency spectrograms.First,a convolutional neural network‐based diagnosis model that utilises these spectrograms is developed to identify whether there is any broken propeller involved in a specific drone flight.Additionally,the authors employ Monte Carlo dropout sampling to obtain the inconsistency of diagnostic results and compute the mean probability score vector's entropy(uncertainty)as another factor to diagnose the drone flight.Next,to reduce data dependence on different drone types,the convolutional neural network‐based diagnosis model is further augmented by transfer learning.That is,the knowledge of a well‐trained diagnosis model is refined by using a small set of data from a different drone.The modified diagnosis model has the ability to detect the broken propeller of the second drone.Thirdly,to reduce the hyperparameters'tuning efforts and reinforce the robustness of the network,Bayesian optimisation takes advantage of the observed diagnosis model performances to construct a Gaussian pro-cess model that allows the acquisition function to choose the optimal network hyper-parameters.The proposed diagnosis framework is validated via real experimental flight tests and has a reasonably high diagnosis accuracy.展开更多
The development of biomimetic aerial robots has emerged as a new solution for studying the flight mechanisms of flying creatures.This study designs a biomimetic robotic butterfly steered via a mass shift mechanism nam...The development of biomimetic aerial robots has emerged as a new solution for studying the flight mechanisms of flying creatures.This study designs a biomimetic robotic butterfly steered via a mass shift mechanism named USTButterfly-II and investigates its flight characteristics using an optical tracking facility.First,a planar fourbar linkage was used to drive the flapping of the designed butterfly-like artificial wings.Next,an innovative tailless steering control method was proposed based on a mass shift mechanism.Finally,the wing kinematics and motion trajectory of the USTButterfly-II were measured using a multi-camera motion capture system,and some difficult-to-measure flapping aerodynamic parameters,such as the instantaneous net lift and thrust coefficients,were determined.These findings present a novel experimental framework that not only provides effective data support for the design and improvement of the robotic butterfly but also benefits the study of biological butterfly flight mechanisms.展开更多
基金supported in part by the National Key Research and Development Program of China(2022YFB4701800 and 2021ZD0114503)the National Natural Science Foundation of China(62103140,U22A2057,62173132,and 62133005)+3 种基金the Hunan Leading Talent of Technological Innovation(2022RC3063)the Top Ten Technical Research Projects of Hunan Province(2024GK1010)the Key Research and Development Program of Hunan Province(2023GK2068)the Science and Technology Innovation Program of Hunan Province(2023RC1049).
文摘New types of aerial robots(NTARs)have found extensive applications in the military,civilian contexts,scientific research,disaster management,and various other domains.Compared with traditional aerial robots,NTARs exhibit a broader range of morphological diversity,locomotion capabilities,and enhanced operational capacities.Therefore,this study defines aerial robots with the four characteristics of morphability,biomimicry,multi-modal locomotion,and manipulator attachment as NTARs.Subsequently,this paper discusses the latest research progress in the materials and manufacturing technology,actuation technology,and perception and control technology of NTARs.Thereafter,the research status of NTAR systems is summarized,focusing on the frontier development and application cases of flapping-wing microair vehicles,perching aerial robots,amphibious robots,and operational aerial robots.Finally,the main challenges presented by NTARs in terms of energy,materials,and perception are analyzed,and the future development trends of NTARs are summarized in terms of size and endurance,mechatronics,and complex scenarios,providing a reference direction for the follow-up exploration of NTARs.
文摘This paper develops a novel hierarchical control strategy for improving the trajectory tracking capability of aerial robots under parameter uncertainties.The hierarchical control strategy is composed of an adaptive sliding mode controller and a model-free iterative sliding mode controller(MFISMC).A position controller is designed based on adaptive sliding mode control(SMC)to safely drive the aerial robot and ensure fast state convergence under external disturbances.Additionally,the MFISMC acts as an attitude controller to estimate the unmodeled dynamics without detailed knowledge of aerial robots.Then,the adaption laws are derived with the Lyapunov theory to guarantee the asymptotic tracking of the system state.Finally,to demonstrate the performance and robustness of the proposed control strategy,numerical simulations are carried out,which are also compared with other conventional strategies,such as proportional-integralderivative(PID),backstepping(BS),and SMC.The simulation results indicate that the proposed hierarchical control strategy can fulfill zero steady-state error and achieve faster convergence compared with conventional strategies.
基金supported by Branding Research Fund by Shibaura Institute of Technology(SIT)。
文摘Since precise self-position estimation is required for autonomous flight of aerial robots, there has been some studies on self-position estimation of indoor aerial robots. In this study, we tackle the self-position estimation problem by mounting a small downward-facing camera on the chassis of an aerial robot. We obtain robot position by sensing the features on the indoor floor.In this work, we used the vertex points(tile corners) where four tiles on a typical tiled floor connected, as an existing feature of the floor. Furthermore, a small lightweight microcontroller is mounted on the robot to perform image processing for the onboard camera. A lightweight image processing algorithm is developed. So, the real-time image processing could be performed by the microcontroller alone which leads to conduct on-board real time tile corner detection. Furthermore, same microcontroller performs control value calculation for flight commanding. The flight commands are implemented based on the detected tile corner information. The above mentioned all devices are mounted on an actual machine, and the effectiveness of the system was investigated.
文摘In aerial robots＇ visual navigation, it is essential yet very difficult to detect the attitude and position of the robots operated in real time. By introducing a new parametric model, the problem can be reduced from almost unmanageable to be partly solved, though not fully, as per the requirement. In this parametric approach, a multi-scale least square method is formulated first. By propagating as well as improving the parameters down from layer to layer of the image pyramid, a new global feature line can then be detected to parameterize the attitude of the robots. Furthermore, this approach paves the way for segmenting the image into distinct parts, which can be realized by deploying a Bayesian classifier on the picture cell level. Comparison with the Hough transform based method in terms of robustness and precision shows that this multi-scale least square algorithm is considerably more robust to noises. Some discussions are also given.
文摘The intersection of Quantum Technologies and Robotics Autonomy is explored in the present paper.The two areas are brought together in establishing an interdisciplinary interface that contributes to advancing the field of system autonomy,and pushes the engineering boundaries beyond the existing techniques.The present research adopts the experimental aspects of quantum entanglement and quantum cryptography,and integrates these established quantum capabilities into distributed robotic platforms,to explore the possibility of achieving increased autonomy for the control of multi-agent robotic systems engaged in cooperative tasks.Experimental quantum capabilities are realized by producing single photons(using spontaneous parametric down-conversion process),polarization of photons,detecting vertical and horizontal polarizations,and single photon detecting/counting.Specifically,such quantum aspects are implemented on network of classical agents,i.e.,classical aerial and ground robots/unmanned systems.With respect to classical systems for robotic applications,leveraging quantum technology is expected to lead to guaranteed security,very fast control and communication,and unparalleled quantum capabilities such as entanglement and quantum superposition that will enable novel applications.
基金funded by the Center for Unmanned Aircraft Systems(C-UAS)a National Science Foundation Industry/University Cooperative Research Center(I/UCRC)under NSF award Numbers IIP-1161036 and CNS-1650547along with significant contributions from C-UAS industry members。
文摘This paper introduces a new algorithm for estimating the relative pose of a moving camera using consecutive frames of a video sequence. State-of-the-art algorithms for calculating the relative pose between two images use matching features to estimate the essential matrix. The essential matrix is then decomposed into the relative rotation and normalized translation between frames. To be robust to noise and feature match outliers, these methods generate a large number of essential matrix hypotheses from randomly selected minimal subsets of feature pairs, and then score these hypotheses on all feature pairs. Alternatively, the algorithm introduced in this paper calculates relative pose hypotheses by directly optimizing the rotation and normalized translation between frames, rather than calculating the essential matrix and then performing the decomposition. The resulting algorithm improves computation time by an order of magnitude. If an inertial measurement unit(IMU) is available, it is used to seed the optimizer, and in addition, we reuse the best hypothesis at each iteration to seed the optimizer thereby reducing the number of relative pose hypotheses that must be generated and scored. These advantages greatly speed up performance and enable the algorithm to run in real-time on low cost embedded hardware. We show application of our algorithm to visual multi-target tracking(MTT) in the presence of parallax and demonstrate its real-time performance on a 640 × 480 video sequence captured on a UAV. Video results are available at https://youtu.be/Hh K-p2 h XNn U.
基金This research work has been partially supported by the Spanish Ministry of Economy and Competitiveness through the project VA4UAV(Visual autonomy for UAV in Dynamic Environments),reference DPI2014-60139-RThe authors would like to thank the members of our research group CVAR(Computer Vision and Aerial Robotics)for their help in software programming and evaluation with real flights:David Palacios,Adrian Diaz-Moreno,Guillermo de Fermin,Alberto Camporredondo and Carlos Valencia.
文摘Purpose–The purpose of this paper is to describe the specification language TML for adaptive mission plans that the authors designed and implemented for the open-source framework Aerostack for aerial robotics.Design/methodology/approach–The TML language combines a task-based hierarchical approach together with a more flexible representation,rule-based reactive planning,to facilitate adaptability.This approach includes additional notions that abstract programming details.The authors built an interpreter integrated in the software framework Aerostack.The interpreter was validated with flight experiments for multi-robot missions in dynamic environments.Findings–The experiments proved that the TML language is easy to use and expressive enough to formulate adaptive missions in dynamic environments.The experiments also showed that the TML interpreter is efficient to execute multi-robot aerial missions and reusable for different platforms.The TML interpreter is able to verify the mission plan before its execution,which increases robustness and safety,avoiding the execution of certain plans that are not feasible.Originality/value–One of the main contributions of this work is the availability of a reliable solution to specify aerial mission plans,integrated in an active open-source project with periodic releases.To the best knowledge of the authors,there are not solutions similar to this in other active open-source projects.As additional contributions,TML uses an original combination of representations for adaptive mission plans(i.e.task trees with original abstract notions and rule-based reactive planning)together with the demonstration of its adequacy for aerial robotics.
基金This material is based upon the work that is partially supported by the National Science Foundation‐USA under Grant No.2046481.
文摘Drones have increasingly collaborated with human workers in some workspaces,such as warehouses.The failure of a drone flight may bring potential risks to human beings'life safety during some aerial tasks.One of the most common flight failures is triggered by damaged propellers.To quickly detect physical damage to propellers,recognise risky flights,and provide early warnings to surrounding human workers,a new and compre-hensive fault diagnosis framework is presented that uses only the audio caused by pro-peller rotation without accessing any flight data.The diagnosis framework includes three components:leverage convolutional neural networks,transfer learning,and Bayesian optimisation.Particularly,the audio signal from an actual flight is collected and trans-ferred into time–frequency spectrograms.First,a convolutional neural network‐based diagnosis model that utilises these spectrograms is developed to identify whether there is any broken propeller involved in a specific drone flight.Additionally,the authors employ Monte Carlo dropout sampling to obtain the inconsistency of diagnostic results and compute the mean probability score vector's entropy(uncertainty)as another factor to diagnose the drone flight.Next,to reduce data dependence on different drone types,the convolutional neural network‐based diagnosis model is further augmented by transfer learning.That is,the knowledge of a well‐trained diagnosis model is refined by using a small set of data from a different drone.The modified diagnosis model has the ability to detect the broken propeller of the second drone.Thirdly,to reduce the hyperparameters'tuning efforts and reinforce the robustness of the network,Bayesian optimisation takes advantage of the observed diagnosis model performances to construct a Gaussian pro-cess model that allows the acquisition function to choose the optimal network hyper-parameters.The proposed diagnosis framework is validated via real experimental flight tests and has a reasonably high diagnosis accuracy.
基金supported by the National Natural Science Foundation of China(62225304,61933001,and 62173031)the Beijing Municipal Natural Science Foundation,China(JQ20026)the Beijing Top Discipline for Artificial Intelligent Science and Engineering,University of Science and Technology Beijing,China.
文摘The development of biomimetic aerial robots has emerged as a new solution for studying the flight mechanisms of flying creatures.This study designs a biomimetic robotic butterfly steered via a mass shift mechanism named USTButterfly-II and investigates its flight characteristics using an optical tracking facility.First,a planar fourbar linkage was used to drive the flapping of the designed butterfly-like artificial wings.Next,an innovative tailless steering control method was proposed based on a mass shift mechanism.Finally,the wing kinematics and motion trajectory of the USTButterfly-II were measured using a multi-camera motion capture system,and some difficult-to-measure flapping aerodynamic parameters,such as the instantaneous net lift and thrust coefficients,were determined.These findings present a novel experimental framework that not only provides effective data support for the design and improvement of the robotic butterfly but also benefits the study of biological butterfly flight mechanisms.
