This paper presents a holistic multimodal approach for real-time anomaly detection and classification in largescale photovoltaic plants.The approach encompasses segmentation,geolocation,and classification of individua...This paper presents a holistic multimodal approach for real-time anomaly detection and classification in largescale photovoltaic plants.The approach encompasses segmentation,geolocation,and classification of individual photovoltaic modules.A fine-tuned Yolov7 model was trained for the individual module’s segmentation of both modalities;RGB and IR images.The localization of individual solar panels relies on photogrammetric measurements to facilitate maintenance operations.The localization process also links extracted images of the same panel using their geographical coordinates and preprocesses them for the multimodal model input.The study also focuses on optimizing pre-trained models using Bayesian search to improve and fine-tune them with our dataset.The dataset was collected from different systems and technologies within our research platform.It has been curated into 1841 images and classified into five anomaly classes.Grad-CAM,an explainable AI tool,is utilized to compare the use of multimodality to a single modality.Finally,for real-time optimization,the ONNX format was used to optimize the model further for deployment in real-time.The improved ConvNext-Tiny model performed well in both modalities,with 99%precision,recall,and F1-score for binary classification and 85%for multi-class classification.In terms of latency,the segmentation models have an inference time of 14 ms and 12 ms for RGB and IR images and 24 ms for detection and classification.The proposed holistic approach includes a built-in feedback loop to ensure the model’s robustness against domain shifts in the production environment.展开更多
Nowadays,the use of Machine Learning(ML)onboard Earth Observation(EO)satellites has been investigated for a plethora of applications relying on multispectral and hyperspectral imaging.Traditionally,these studies have ...Nowadays,the use of Machine Learning(ML)onboard Earth Observation(EO)satellites has been investigated for a plethora of applications relying on multispectral and hyperspectral imaging.Traditionally,these studies have heavily relied on high-end data products,subjected to extensive pre-processing chains natively designed to be executed on the ground.However,replicating such algorithms onboard EO satellites poses significant challenges due to their computational intensity and need for additional metadata,which are typically unavailable on board.Because of that,current missions exploring onboard ML models implement simplified but still complex processing chains that imitate their on-ground counterparts.Despite these advancements,the potential of ML models to process raw satellite data directly remains largely unexplored.To fill this gap,this paper investigates the feasibility of applying ML models directly to Sentinel-2 raw data to perform thermal hotspot classification.This approach significantly limits the processing steps to simple and lightweight algorithms to achieve real-time processing of data with low power consumption.To this aim,we present an end-to-end(E2E)pipeline to create a binary classification map of Sentinel-2 raw granules,where each point suggests the absence/presence of a thermal anomaly in a square area of 2.5 km.To this aim,lightweight coarse spatial registration is applied to register three different bands,and an EfficientNetlite0 model is used to perform the classification of the various bands.The trained models achieve an average Matthew’s correlation coefficient(MCC)score of 0.854(on 5 seeds)and a maximum MCC of 0.90 on a geographically tripartite dataset of cropped images from the THRawS dataset.The proposed E2E pipeline is capable of processing a Sentinel-2 granule in 1.8 s and within 6.4 W peak power on a combination of Raspberry PI 4 and CogniSat-XE2 board,demonstrating real-time performance.展开更多
This paper addresses the significance of preprocessing big data collected during a tunnel boring machine(TBM)excavation before it is used for machine learning on various TBM performance predictions.The research work i...This paper addresses the significance of preprocessing big data collected during a tunnel boring machine(TBM)excavation before it is used for machine learning on various TBM performance predictions.The research work is based on two water diversion tunneling projects that cover 29.52 km and 17051 boring cycles.It has been found that the penetration rate calculated from the raw measured penetration distances exhibits more random behavior owing to their percussive and vibratory behavior of the cutterhead.A moving average method to process the negative instantaneous velocities and a noise reduction filter to deal with signals with abnormal frequencies have been recommended.An index called the drilling efficiency index is introduced to assess the relationships between the mechanical parameters in a boring cycle,whose linear regression coefficient R^(2)is taken for a preliminary investigation of possible problems requiring preprocessing.The research work defines the irrelevant data whose errors are caused by human or mechanical mistakes,and therefore should be cleaned or amended.These irrelevant data can be divided into five categories:(1)premature cycles,(2)sensor defects,(3)mechanical defects,(4)human interruption,and(5)missing files.A program TBM-Processing has been coded for the recognition and classification of these categories.PDF books generated by the program have been uploaded at GitHub to encourage discussions,collaboration,and upgrading of the data processing work with our peers.展开更多
基金Digitalized PV plant project financed by OCP group[Reference:Digital and smart photovoltaic power plant].
文摘This paper presents a holistic multimodal approach for real-time anomaly detection and classification in largescale photovoltaic plants.The approach encompasses segmentation,geolocation,and classification of individual photovoltaic modules.A fine-tuned Yolov7 model was trained for the individual module’s segmentation of both modalities;RGB and IR images.The localization of individual solar panels relies on photogrammetric measurements to facilitate maintenance operations.The localization process also links extracted images of the same panel using their geographical coordinates and preprocesses them for the multimodal model input.The study also focuses on optimizing pre-trained models using Bayesian search to improve and fine-tune them with our dataset.The dataset was collected from different systems and technologies within our research platform.It has been curated into 1841 images and classified into five anomaly classes.Grad-CAM,an explainable AI tool,is utilized to compare the use of multimodality to a single modality.Finally,for real-time optimization,the ONNX format was used to optimize the model further for deployment in real-time.The improved ConvNext-Tiny model performed well in both modalities,with 99%precision,recall,and F1-score for binary classification and 85%for multi-class classification.In terms of latency,the segmentation models have an inference time of 14 ms and 12 ms for RGB and IR images and 24 ms for detection and classification.The proposed holistic approach includes a built-in feedback loop to ensure the model’s robustness against domain shifts in the production environment.
文摘Nowadays,the use of Machine Learning(ML)onboard Earth Observation(EO)satellites has been investigated for a plethora of applications relying on multispectral and hyperspectral imaging.Traditionally,these studies have heavily relied on high-end data products,subjected to extensive pre-processing chains natively designed to be executed on the ground.However,replicating such algorithms onboard EO satellites poses significant challenges due to their computational intensity and need for additional metadata,which are typically unavailable on board.Because of that,current missions exploring onboard ML models implement simplified but still complex processing chains that imitate their on-ground counterparts.Despite these advancements,the potential of ML models to process raw satellite data directly remains largely unexplored.To fill this gap,this paper investigates the feasibility of applying ML models directly to Sentinel-2 raw data to perform thermal hotspot classification.This approach significantly limits the processing steps to simple and lightweight algorithms to achieve real-time processing of data with low power consumption.To this aim,we present an end-to-end(E2E)pipeline to create a binary classification map of Sentinel-2 raw granules,where each point suggests the absence/presence of a thermal anomaly in a square area of 2.5 km.To this aim,lightweight coarse spatial registration is applied to register three different bands,and an EfficientNetlite0 model is used to perform the classification of the various bands.The trained models achieve an average Matthew’s correlation coefficient(MCC)score of 0.854(on 5 seeds)and a maximum MCC of 0.90 on a geographically tripartite dataset of cropped images from the THRawS dataset.The proposed E2E pipeline is capable of processing a Sentinel-2 granule in 1.8 s and within 6.4 W peak power on a combination of Raspberry PI 4 and CogniSat-XE2 board,demonstrating real-time performance.
基金support from the National Program on Key Basic Research Project(973 Program,No.2015CB058100)of China and China Railway Engineering Equipment Group Corporationsupported by the Key Research Project of China Institute of Water Resources and Hydropower Research Limited(Grant Nos.HTGE0203A03201900000,HTGE0203A20202000000)Natural Science Foundation of Shaanxi Province(Grant Nos.2019JLZ-13,2019JLP-23).
文摘This paper addresses the significance of preprocessing big data collected during a tunnel boring machine(TBM)excavation before it is used for machine learning on various TBM performance predictions.The research work is based on two water diversion tunneling projects that cover 29.52 km and 17051 boring cycles.It has been found that the penetration rate calculated from the raw measured penetration distances exhibits more random behavior owing to their percussive and vibratory behavior of the cutterhead.A moving average method to process the negative instantaneous velocities and a noise reduction filter to deal with signals with abnormal frequencies have been recommended.An index called the drilling efficiency index is introduced to assess the relationships between the mechanical parameters in a boring cycle,whose linear regression coefficient R^(2)is taken for a preliminary investigation of possible problems requiring preprocessing.The research work defines the irrelevant data whose errors are caused by human or mechanical mistakes,and therefore should be cleaned or amended.These irrelevant data can be divided into five categories:(1)premature cycles,(2)sensor defects,(3)mechanical defects,(4)human interruption,and(5)missing files.A program TBM-Processing has been coded for the recognition and classification of these categories.PDF books generated by the program have been uploaded at GitHub to encourage discussions,collaboration,and upgrading of the data processing work with our peers.
