Nowadays,the use of Machine Learning(ML)onboard Earth Observation(EO)satellites has been investigated for a plethora of applications relying on multispectral and hyperspectral imaging.Traditionally,these studies have ...Nowadays,the use of Machine Learning(ML)onboard Earth Observation(EO)satellites has been investigated for a plethora of applications relying on multispectral and hyperspectral imaging.Traditionally,these studies have heavily relied on high-end data products,subjected to extensive pre-processing chains natively designed to be executed on the ground.However,replicating such algorithms onboard EO satellites poses significant challenges due to their computational intensity and need for additional metadata,which are typically unavailable on board.Because of that,current missions exploring onboard ML models implement simplified but still complex processing chains that imitate their on-ground counterparts.Despite these advancements,the potential of ML models to process raw satellite data directly remains largely unexplored.To fill this gap,this paper investigates the feasibility of applying ML models directly to Sentinel-2 raw data to perform thermal hotspot classification.This approach significantly limits the processing steps to simple and lightweight algorithms to achieve real-time processing of data with low power consumption.To this aim,we present an end-to-end(E2E)pipeline to create a binary classification map of Sentinel-2 raw granules,where each point suggests the absence/presence of a thermal anomaly in a square area of 2.5 km.To this aim,lightweight coarse spatial registration is applied to register three different bands,and an EfficientNetlite0 model is used to perform the classification of the various bands.The trained models achieve an average Matthew’s correlation coefficient(MCC)score of 0.854(on 5 seeds)and a maximum MCC of 0.90 on a geographically tripartite dataset of cropped images from the THRawS dataset.The proposed E2E pipeline is capable of processing a Sentinel-2 granule in 1.8 s and within 6.4 W peak power on a combination of Raspberry PI 4 and CogniSat-XE2 board,demonstrating real-time performance.展开更多
With the prevalence of Internet of Things(loT)devices,data collection has the potential to improve people's lives and create a significant value.However,it also exposes sensitive information,which leads to privacy...With the prevalence of Internet of Things(loT)devices,data collection has the potential to improve people's lives and create a significant value.However,it also exposes sensitive information,which leads to privacy risks.An approach called N-source anonymity has been used for privacy preservation in raw data collection,but most of the existing schemes do not have a balanced efficiency and robustness.In this work,a lightweight and efficient raw data collection scheme is proposed.The proposed scheme can not only collect data from the original users but also protect their privacy.Besides,the proposed scheme can resist user poisoning attacks,and the use of the reward method can motivate users to actively provide data.Analysis and simulation indicate that the proposed scheme is safe against poison attacks.Additionally,the proposed scheme has better performance in terms of computation and communication overhead compared to existing methods.High efficiency and appropriate incentive mechanisms indicate that the scheme is practical for IoT systems.展开更多
Photonuclear data are increasingly used in fundamental nuclear research and technological applications.These data are generated using advanced γ-ray sources.The Shanghai laser electron gamma source(SLEGS)is a new las...Photonuclear data are increasingly used in fundamental nuclear research and technological applications.These data are generated using advanced γ-ray sources.The Shanghai laser electron gamma source(SLEGS)is a new laser Compton scattering γ-ray source at the Shanghai Synchrotron Radiation Facility.It delivers energy-tunable,quasi-monoenergetic gamma beams for high-precision photonuclear measurements.This paper presents the flat-efficiency detector(FED)array at SLEGS and its application in photoneutron cross-section measurements.Systematic uncertainties of the FED array were determined to be 3.02%through calibration with a ^(252)Cf neutron source.Using ^(197)Au and ^(159)Tb as representative nuclei,we demonstrate the format and processing methodology for raw photoneutron data.The results validate SLEGS’capability for high-precision photoneutron measurements.展开更多
The analysis of ancient genomics provides opportunities to explore human population history across both temporal and geographic dimensions(Haak et al.,2015;Wang et al.,2021,2024)to enhance the accessibility and utilit...The analysis of ancient genomics provides opportunities to explore human population history across both temporal and geographic dimensions(Haak et al.,2015;Wang et al.,2021,2024)to enhance the accessibility and utility of these ancient genomic datasets,a range of databases and advanced statistical models have been developed,including the Allen Ancient DNA Resource(AADR)(Mallick et al.,2024)and AdmixTools(Patterson et al.,2012).While upstream processes such as sequencing and raw data processing have been streamlined by resources like the AADR,the downstream analysis of these datasets-encompassing population genetics inference and spatiotemporal interpretation-remains a significant challenge.The AADR provides a unified collection of published ancient DNA(aDNA)data,yet its file-based format and reliance on command-line tools,such as those in Admix-Tools(Patterson et al.,2012),require advanced computational expertise for effective exploration and analysis.These requirements can present significant challenges forresearchers lackingadvanced computational expertise,limiting the accessibility and broader application of these valuable genomic resources.展开更多
在长期的计算机视觉技术发展过程中,研究者们主要处理的对象是通过图像信号处理器(Image Signal Processor,ISP)处理后的标准RGB图像。这类图像体积小,方便使用和网络传播,因而在许多传统应用场景中得到广泛应用。然而,在低光照或极端...在长期的计算机视觉技术发展过程中,研究者们主要处理的对象是通过图像信号处理器(Image Signal Processor,ISP)处理后的标准RGB图像。这类图像体积小,方便使用和网络传播,因而在许多传统应用场景中得到广泛应用。然而,在低光照或极端成像条件下,这类经过压缩和处理的图像往往因模糊、量化等不可逆操作,导致细节丢失,限制了其性能表现。为应对这些挑战,越来越多的研究开始关注直接处理相机传感器输出的RAW图像。RAW图像未经过复杂的ISP处理,具有线性响应、高比特深度和无损压缩的特点,能够保留更多的原始感光信息。这些特性使其在低光、高动态范围以及复杂视觉场景中表现出优异的灵活性和潜力。在近年来的研究中,RAW图像处理技术取得了显著进展,其应用已从高质量图像与视频的获取、去噪与增强,扩展到目标识别、场景理解等计算机视觉任务。相比传统RGB图像,RAW图像处理能够更好地保留细节信息,并在特定条件下显著提升视觉任务的精度和鲁棒性。此外,随着深度学习技术的发展,基于RAW数据的端到端模型设计成为了新的研究方向,能够充分利用图像中的原始信号信息来提升视觉处理效果。本文系统性地综述了RAW图像处理技术的最新进展,并探讨了这些技术在计算机视觉各领域中的应用。同时,本文还展望了未来的发展趋势,特别是RAW图像数据在更复杂场景下的应用潜力,为相关研究者和从业者提供了有价值的参考和启示。展开更多
When the saturation degree （SD） of space-borne SAR raw data is high, the performance of conventional block adaptive quantization （BAQ） deteriorates obviously. In order to overcome the drawback, this paper studies ...When the saturation degree （SD） of space-borne SAR raw data is high, the performance of conventional block adaptive quantization （BAQ） deteriorates obviously. In order to overcome the drawback, this paper studies the mapping between the average signal magnitude （ASM） and the standard deviation of the input signal （SDIS） to the A/D from the original reference. Then, it points out the mistake of the mapping and introduces the concept of the standard deviation of the output signal （SDOS） from the A/D. After that, this paper educes the mapping between the ASM and SDOS from the A/D. Monte-Carlo experiment shows that none of the above two mappings is the optimal in the whole set of SD. Thus, this paper proposes the concept of piecewise linear mapping and the searching algorithm in the whole set of SD. According to the linear part, this paper gives the certification and analytical value of k and for nonlinear part, and utilizes the searching algorithm mentioned above to search the corresponding value of k. Experimental results based on simulated data and real data show that the performance of new algorithm is better than conventional BAQ when raw data is in heavy SD.展开更多
The Synthetic Aperture Radar(SAR)raw data generator is required to the evaluation of focusing algorithms,moving target analysis,and hardware design.The time-domain SAR simulator can generate the accurate raw data but ...The Synthetic Aperture Radar(SAR)raw data generator is required to the evaluation of focusing algorithms,moving target analysis,and hardware design.The time-domain SAR simulator can generate the accurate raw data but it needs much time.The frequency-domain simulator not only increases the efficiency but also considers the trajectory deviations of the radar.In addition,the raw signal of the extended scene included static and moving targets can be generated by some frequency-domain simulators.However,the existing simulators concentrate on the raw signal simulation of the static extended scene and moving targets at uniform speed mostly.As for the issue,the two-dimensional signal spectrum of moving targets with constant acceleration can be derived accurately based on the geometric model of a side-looking SAR and reversion of series.And a frequency-domain algorithm for SAR echo signal simulation is presented based on the two-dimensional signal spectrum.The raw data generated with proposed method is verified by several simulation experiments.In addition to reveal the efficiency of the presented frequency-domain SAR scene simulator,the computational complexity of the proposed method is compared with the time-domain approach using the complex multiplication.Numerical results demonstrate that the present method can reduce the computational time significantly without accuracy loss while simulating SAR raw data.展开更多
Phase errors in synthetic aperture sonar (SAS) imaging must be reduced to less than one eighth of a wavelength so as to avoid image destruction. Most of the phase errors occur as a result of platform motion errors, fo...Phase errors in synthetic aperture sonar (SAS) imaging must be reduced to less than one eighth of a wavelength so as to avoid image destruction. Most of the phase errors occur as a result of platform motion errors, for example, sway yaw and surge that are the most important error sources. The phase error of a wide band synthetic aperture sonar is modeled and solutions to sway yaw and surge motion estimation based on the raw sonar echo data with a Displaced Phase Center Antenna (DPCA) method are proposed and their implementations are detailed in this paper. It is shown that the sway estimates can be obtained from the correlation lag and phase difference between the returns at coincident phase centers. An estimate of yaw is also possible if such a technique is applied to more than one overlapping phase center positions. Surge estimates can be obtained by identifying pairs of phase centers with a maximum correlation coefficient. The method works only if the platform velocity is low enough such that a number of phase centers from adjacent pings overlap.展开更多
Dear Editor,I would like to congratulate Mamsen et al.i on their extensive and scientifically valuable work.I analyzed their raw data presented in Table 1 of the original article from a different perspective and disco...Dear Editor,I would like to congratulate Mamsen et al.i on their extensive and scientifically valuable work.I analyzed their raw data presented in Table 1 of the original article from a different perspective and discovered an effect not mentioned in the article.My analysis showed that luteinizing hormone(LH)levels are significantly lower in patients at high infertility risk(HIR),whose testes lack A dark(Ad)spermatogonia and display an abnormal ratio of germ cells per crosssectional tubule(G/T).展开更多
An accurate and efficient Synthetic Aperture Radar(SAR)raw data generator is of considerable value for testing system parameters and verifying imaging algorithms.Nevertheless,the existing simulator cannot exactly hand...An accurate and efficient Synthetic Aperture Radar(SAR)raw data generator is of considerable value for testing system parameters and verifying imaging algorithms.Nevertheless,the existing simulator cannot exactly handle the case of the fast moving targets in high squint geometry.As for the issue,the analytical expression for the two Dimensional(2-D)signal spectrum of moving targets is derived and a fast raw echo simulation method is proposed in this study.The proposed simulator can accommodate the moving targets in the high squint geometry,whose processing steps of the simulation are given in detail and its computational complexity is analyzed.The simulation data for static and moving targets are processed and analyzed,and the results are given to validate the effectiveness of the proposed approach.展开更多
Full electronic automation in stock exchanges has recently become popular,generat-ing high-frequency intraday data and motivating the development of near real-time price forecasting methods.Machine learning algorithms...Full electronic automation in stock exchanges has recently become popular,generat-ing high-frequency intraday data and motivating the development of near real-time price forecasting methods.Machine learning algorithms are widely applied to mid-price stock predictions.Processing raw data as inputs for prediction models(e.g.,data thinning and feature engineering)can primarily affect the performance of the prediction methods.However,researchers rarely discuss this topic.This motivated us to propose three novel modelling strategies for processing raw data.We illustrate how our novel modelling strategies improve forecasting performance by analyzing high-frequency data of the Dow Jones 30 component stocks.In these experiments,our strategies often lead to statistically significant improvement in predictions.The three strategies improve the F1 scores of the SVM models by 0.056,0.087,and 0.016,respectively.展开更多
文摘Nowadays,the use of Machine Learning(ML)onboard Earth Observation(EO)satellites has been investigated for a plethora of applications relying on multispectral and hyperspectral imaging.Traditionally,these studies have heavily relied on high-end data products,subjected to extensive pre-processing chains natively designed to be executed on the ground.However,replicating such algorithms onboard EO satellites poses significant challenges due to their computational intensity and need for additional metadata,which are typically unavailable on board.Because of that,current missions exploring onboard ML models implement simplified but still complex processing chains that imitate their on-ground counterparts.Despite these advancements,the potential of ML models to process raw satellite data directly remains largely unexplored.To fill this gap,this paper investigates the feasibility of applying ML models directly to Sentinel-2 raw data to perform thermal hotspot classification.This approach significantly limits the processing steps to simple and lightweight algorithms to achieve real-time processing of data with low power consumption.To this aim,we present an end-to-end(E2E)pipeline to create a binary classification map of Sentinel-2 raw granules,where each point suggests the absence/presence of a thermal anomaly in a square area of 2.5 km.To this aim,lightweight coarse spatial registration is applied to register three different bands,and an EfficientNetlite0 model is used to perform the classification of the various bands.The trained models achieve an average Matthew’s correlation coefficient(MCC)score of 0.854(on 5 seeds)and a maximum MCC of 0.90 on a geographically tripartite dataset of cropped images from the THRawS dataset.The proposed E2E pipeline is capable of processing a Sentinel-2 granule in 1.8 s and within 6.4 W peak power on a combination of Raspberry PI 4 and CogniSat-XE2 board,demonstrating real-time performance.
基金supported in part by the National Natural Science Foundation of China(62072133)the Innovation Project of Guangxi Graduate Education(YCSW2022279)Wenzhou Science and Technology Plan(2023ZW0013).
文摘With the prevalence of Internet of Things(loT)devices,data collection has the potential to improve people's lives and create a significant value.However,it also exposes sensitive information,which leads to privacy risks.An approach called N-source anonymity has been used for privacy preservation in raw data collection,but most of the existing schemes do not have a balanced efficiency and robustness.In this work,a lightweight and efficient raw data collection scheme is proposed.The proposed scheme can not only collect data from the original users but also protect their privacy.Besides,the proposed scheme can resist user poisoning attacks,and the use of the reward method can motivate users to actively provide data.Analysis and simulation indicate that the proposed scheme is safe against poison attacks.Additionally,the proposed scheme has better performance in terms of computation and communication overhead compared to existing methods.High efficiency and appropriate incentive mechanisms indicate that the scheme is practical for IoT systems.
基金supported by National Key Research and Development Program of China(Nos.2022YFA1602404 and 2023YFA1606901)the National Natural Science Foundation of China(Nos.12275338,12388102,and U2441221)the Key Laboratory of Nuclear Data Foundation(JCKY2022201C152).
文摘Photonuclear data are increasingly used in fundamental nuclear research and technological applications.These data are generated using advanced γ-ray sources.The Shanghai laser electron gamma source(SLEGS)is a new laser Compton scattering γ-ray source at the Shanghai Synchrotron Radiation Facility.It delivers energy-tunable,quasi-monoenergetic gamma beams for high-precision photonuclear measurements.This paper presents the flat-efficiency detector(FED)array at SLEGS and its application in photoneutron cross-section measurements.Systematic uncertainties of the FED array were determined to be 3.02%through calibration with a ^(252)Cf neutron source.Using ^(197)Au and ^(159)Tb as representative nuclei,we demonstrate the format and processing methodology for raw photoneutron data.The results validate SLEGS’capability for high-precision photoneutron measurements.
基金by the National Key Research and Development Program of China(2023YFC3303701-02 and 2024YFC3306701)the National Natural Science Foundation of China(T2425014 and 32270667)+3 种基金the Natural Science Foundation of Fujian Province of China(2023J06013)the Major Project of the National Social Science Foundation of China granted to Chuan-Chao Wang(21&ZD285)Open Research Fund of State Key Laboratory of Genetic Engineering at Fudan University(SKLGE-2310)Open Research Fund of Forensic Genetics Key Laboratory of the Ministry of Public Security(2023FGKFKT07).
文摘The analysis of ancient genomics provides opportunities to explore human population history across both temporal and geographic dimensions(Haak et al.,2015;Wang et al.,2021,2024)to enhance the accessibility and utility of these ancient genomic datasets,a range of databases and advanced statistical models have been developed,including the Allen Ancient DNA Resource(AADR)(Mallick et al.,2024)and AdmixTools(Patterson et al.,2012).While upstream processes such as sequencing and raw data processing have been streamlined by resources like the AADR,the downstream analysis of these datasets-encompassing population genetics inference and spatiotemporal interpretation-remains a significant challenge.The AADR provides a unified collection of published ancient DNA(aDNA)data,yet its file-based format and reliance on command-line tools,such as those in Admix-Tools(Patterson et al.,2012),require advanced computational expertise for effective exploration and analysis.These requirements can present significant challenges forresearchers lackingadvanced computational expertise,limiting the accessibility and broader application of these valuable genomic resources.
文摘在长期的计算机视觉技术发展过程中,研究者们主要处理的对象是通过图像信号处理器(Image Signal Processor,ISP)处理后的标准RGB图像。这类图像体积小,方便使用和网络传播,因而在许多传统应用场景中得到广泛应用。然而,在低光照或极端成像条件下,这类经过压缩和处理的图像往往因模糊、量化等不可逆操作,导致细节丢失,限制了其性能表现。为应对这些挑战,越来越多的研究开始关注直接处理相机传感器输出的RAW图像。RAW图像未经过复杂的ISP处理,具有线性响应、高比特深度和无损压缩的特点,能够保留更多的原始感光信息。这些特性使其在低光、高动态范围以及复杂视觉场景中表现出优异的灵活性和潜力。在近年来的研究中,RAW图像处理技术取得了显著进展,其应用已从高质量图像与视频的获取、去噪与增强,扩展到目标识别、场景理解等计算机视觉任务。相比传统RGB图像,RAW图像处理能够更好地保留细节信息,并在特定条件下显著提升视觉任务的精度和鲁棒性。此外,随着深度学习技术的发展,基于RAW数据的端到端模型设计成为了新的研究方向,能够充分利用图像中的原始信号信息来提升视觉处理效果。本文系统性地综述了RAW图像处理技术的最新进展,并探讨了这些技术在计算机视觉各领域中的应用。同时,本文还展望了未来的发展趋势,特别是RAW图像数据在更复杂场景下的应用潜力,为相关研究者和从业者提供了有价值的参考和启示。
文摘When the saturation degree （SD） of space-borne SAR raw data is high, the performance of conventional block adaptive quantization （BAQ） deteriorates obviously. In order to overcome the drawback, this paper studies the mapping between the average signal magnitude （ASM） and the standard deviation of the input signal （SDIS） to the A/D from the original reference. Then, it points out the mistake of the mapping and introduces the concept of the standard deviation of the output signal （SDOS） from the A/D. After that, this paper educes the mapping between the ASM and SDOS from the A/D. Monte-Carlo experiment shows that none of the above two mappings is the optimal in the whole set of SD. Thus, this paper proposes the concept of piecewise linear mapping and the searching algorithm in the whole set of SD. According to the linear part, this paper gives the certification and analytical value of k and for nonlinear part, and utilizes the searching algorithm mentioned above to search the corresponding value of k. Experimental results based on simulated data and real data show that the performance of new algorithm is better than conventional BAQ when raw data is in heavy SD.
基金The work was supported by the Natural Science Foundation of Shandong Province,China.(Grant No.ZR2017BF032)。
文摘The Synthetic Aperture Radar(SAR)raw data generator is required to the evaluation of focusing algorithms,moving target analysis,and hardware design.The time-domain SAR simulator can generate the accurate raw data but it needs much time.The frequency-domain simulator not only increases the efficiency but also considers the trajectory deviations of the radar.In addition,the raw signal of the extended scene included static and moving targets can be generated by some frequency-domain simulators.However,the existing simulators concentrate on the raw signal simulation of the static extended scene and moving targets at uniform speed mostly.As for the issue,the two-dimensional signal spectrum of moving targets with constant acceleration can be derived accurately based on the geometric model of a side-looking SAR and reversion of series.And a frequency-domain algorithm for SAR echo signal simulation is presented based on the two-dimensional signal spectrum.The raw data generated with proposed method is verified by several simulation experiments.In addition to reveal the efficiency of the presented frequency-domain SAR scene simulator,the computational complexity of the proposed method is compared with the time-domain approach using the complex multiplication.Numerical results demonstrate that the present method can reduce the computational time significantly without accuracy loss while simulating SAR raw data.
文摘Phase errors in synthetic aperture sonar (SAS) imaging must be reduced to less than one eighth of a wavelength so as to avoid image destruction. Most of the phase errors occur as a result of platform motion errors, for example, sway yaw and surge that are the most important error sources. The phase error of a wide band synthetic aperture sonar is modeled and solutions to sway yaw and surge motion estimation based on the raw sonar echo data with a Displaced Phase Center Antenna (DPCA) method are proposed and their implementations are detailed in this paper. It is shown that the sway estimates can be obtained from the correlation lag and phase difference between the returns at coincident phase centers. An estimate of yaw is also possible if such a technique is applied to more than one overlapping phase center positions. Surge estimates can be obtained by identifying pairs of phase centers with a maximum correlation coefficient. The method works only if the platform velocity is low enough such that a number of phase centers from adjacent pings overlap.
文摘Dear Editor,I would like to congratulate Mamsen et al.i on their extensive and scientifically valuable work.I analyzed their raw data presented in Table 1 of the original article from a different perspective and discovered an effect not mentioned in the article.My analysis showed that luteinizing hormone(LH)levels are significantly lower in patients at high infertility risk(HIR),whose testes lack A dark(Ad)spermatogonia and display an abnormal ratio of germ cells per crosssectional tubule(G/T).
文摘An accurate and efficient Synthetic Aperture Radar(SAR)raw data generator is of considerable value for testing system parameters and verifying imaging algorithms.Nevertheless,the existing simulator cannot exactly handle the case of the fast moving targets in high squint geometry.As for the issue,the analytical expression for the two Dimensional(2-D)signal spectrum of moving targets is derived and a fast raw echo simulation method is proposed in this study.The proposed simulator can accommodate the moving targets in the high squint geometry,whose processing steps of the simulation are given in detail and its computational complexity is analyzed.The simulation data for static and moving targets are processed and analyzed,and the results are given to validate the effectiveness of the proposed approach.
基金Canada Research Chair(950231363,XZ),Natural Sciences and Engineering Research Council of Canada(NSERC)Discovery Grants(RGPIN-20203530,LX)the Social Sciences and Humanities Research Council of Canada(SSHRC)Insight Development Grants(430-2018-00557,KX).
文摘Full electronic automation in stock exchanges has recently become popular,generat-ing high-frequency intraday data and motivating the development of near real-time price forecasting methods.Machine learning algorithms are widely applied to mid-price stock predictions.Processing raw data as inputs for prediction models(e.g.,data thinning and feature engineering)can primarily affect the performance of the prediction methods.However,researchers rarely discuss this topic.This motivated us to propose three novel modelling strategies for processing raw data.We illustrate how our novel modelling strategies improve forecasting performance by analyzing high-frequency data of the Dow Jones 30 component stocks.In these experiments,our strategies often lead to statistically significant improvement in predictions.The three strategies improve the F1 scores of the SVM models by 0.056,0.087,and 0.016,respectively.
