期刊文献+
共找到25,195篇文章
< 1 2 250 >
每页显示 20 50 100
Modeling and Performance Evaluation of Streaming Data Processing System in IoT Architecture
1
作者 Feng Zhu Kailin Wu Jie Ding 《Computers, Materials & Continua》 2025年第5期2573-2598,共26页
With the widespread application of Internet of Things(IoT)technology,the processing of massive realtime streaming data poses significant challenges to the computational and data-processing capabilities of systems.Alth... With the widespread application of Internet of Things(IoT)technology,the processing of massive realtime streaming data poses significant challenges to the computational and data-processing capabilities of systems.Although distributed streaming data processing frameworks such asApache Flink andApache Spark Streaming provide solutions,meeting stringent response time requirements while ensuring high throughput and resource utilization remains an urgent problem.To address this,the study proposes a formal modeling approach based on Performance Evaluation Process Algebra(PEPA),which abstracts the core components and interactions of cloud-based distributed streaming data processing systems.Additionally,a generic service flow generation algorithmis introduced,enabling the automatic extraction of service flows fromthe PEPAmodel and the computation of key performance metrics,including response time,throughput,and resource utilization.The novelty of this work lies in the integration of PEPA-based formal modeling with the service flow generation algorithm,bridging the gap between formal modeling and practical performance evaluation for IoT systems.Simulation experiments demonstrate that optimizing the execution efficiency of components can significantly improve system performance.For instance,increasing the task execution rate from 10 to 100 improves system performance by 9.53%,while further increasing it to 200 results in a 21.58%improvement.However,diminishing returns are observed when the execution rate reaches 500,with only a 0.42%gain.Similarly,increasing the number of TaskManagers from 10 to 20 improves response time by 18.49%,but the improvement slows to 6.06% when increasing from 20 to 50,highlighting the importance of co-optimizing component efficiency and resource management to achieve substantial performance gains.This study provides a systematic framework for analyzing and optimizing the performance of IoT systems for large-scale real-time streaming data processing.The proposed approach not only identifies performance bottlenecks but also offers insights into improving system efficiency under different configurations and workloads. 展开更多
关键词 System modeling performance evaluation streaming data process IoT system PEPA
在线阅读 下载PDF
Enhancing the data processing speed of a deep-learning-based three-dimensional single molecule localization algorithm (FD-DeepLoc) with a combination of feature compression and pipeline programming
2
作者 Shuhao Guo Jiaxun Lin +1 位作者 Yingjun Zhang Zhen-Li Huang 《Journal of Innovative Optical Health Sciences》 2025年第2期150-160,共11页
Three-dimensional(3D)single molecule localization microscopy(SMLM)plays an important role in biomedical applications,but its data processing is very complicated.Deep learning is a potential tool to solve this problem.... Three-dimensional(3D)single molecule localization microscopy(SMLM)plays an important role in biomedical applications,but its data processing is very complicated.Deep learning is a potential tool to solve this problem.As the state of art 3D super-resolution localization algorithm based on deep learning,FD-DeepLoc algorithm reported recently still has a gap with the expected goal of online image processing,even though it has greatly improved the data processing throughput.In this paper,a new algorithm Lite-FD-DeepLoc is developed on the basis of FD-DeepLoc algorithm to meet the online image processing requirements of 3D SMLM.This new algorithm uses the feature compression method to reduce the parameters of the model,and combines it with pipeline programming to accelerate the inference process of the deep learning model.The simulated data processing results show that the image processing speed of Lite-FD-DeepLoc is about twice as fast as that of FD-DeepLoc with a slight decrease in localization accuracy,which can realize real-time processing of 256×256 pixels size images.The results of biological experimental data processing imply that Lite-FD-DeepLoc can successfully analyze the data based on astigmatism and saddle point engineering,and the global resolution of the reconstructed image is equivalent to or even better than FD-DeepLoc algorithm. 展开更多
关键词 Real-time data processing feature compression pipeline programming
原文传递
Automation and parallelization scheme to accelerate pulsar observation data processing
3
作者 Xingnan Zhang Minghui Li 《Astronomical Techniques and Instruments》 2025年第4期226-238,共13页
Previous studies aiming to accelerate data processing have focused on enhancement algorithms,using the graphics processing unit(GPU)to speed up programs,and thread-level parallelism.These methods overlook maximizing t... Previous studies aiming to accelerate data processing have focused on enhancement algorithms,using the graphics processing unit(GPU)to speed up programs,and thread-level parallelism.These methods overlook maximizing the utilization of existing central processing unit(CPU)resources and reducing human and computational time costs via process automation.Accordingly,this paper proposes a scheme,called SSM,that combines“Srun job submission mode”,“Sbatch job submission mode”,and“Monitor function”.The SSM scheme includes three main modules:data management,command management,and resource management.Its core innovations are command splitting and parallel execution.The results show that this method effectively improves CPU utilization and reduces the time required for data processing.In terms of CPU utilization,the average value of this scheme is 89%.In contrast,the average CPU utilizations of“Srun job submission mode”and“Sbatch job submission mode”are significantly lower,at 43%and 52%,respectively.In terms of the data-processing time,SSM testing on the Five-hundred-meter Aperture Spherical radio Telescope(FAST)data requires only 5.5 h,compared with 8 h in the“Srun job submission mode”and 14 h in the“Sbatch job submission mode”.In addition,tests on the FAST and Parkes datasets demonstrate the universality of the SSM scheme,which can process data from different telescopes.The compatibility of the SSM scheme for pulsar searches is verified using 2 days of observational data from the globular cluster M2,with the scheme successfully discovering all published pulsars in M2. 展开更多
关键词 Astronomical data Parallel processing PulsaR Exploration and Search TOolkit(PRESTO) CPU FAST Parkes
在线阅读 下载PDF
A review of test methods for uniaxial compressive strength of rocks:Theory,apparatus and data processing
4
作者 Wei-Qiang Xie Xiao-Li Liu +2 位作者 Xiao-Ping Zhang Quan-Sheng Liu En-ZhiWang 《Journal of Rock Mechanics and Geotechnical Engineering》 2025年第3期1889-1905,共17页
The uniaxial compressive strength(UCS)of rocks is a vital geomechanical parameter widely used for rock mass classification,stability analysis,and engineering design in rock engineering.Various UCS testing methods and ... The uniaxial compressive strength(UCS)of rocks is a vital geomechanical parameter widely used for rock mass classification,stability analysis,and engineering design in rock engineering.Various UCS testing methods and apparatuses have been proposed over the past few decades.The objective of the present study is to summarize the status and development in theories,test apparatuses,data processing of the existing testing methods for UCS measurement.It starts with elaborating the theories of these test methods.Then the test apparatus and development trends for UCS measurement are summarized,followed by a discussion on rock specimens for test apparatus,and data processing methods.Next,the method selection for UCS measurement is recommended.It reveals that the rock failure mechanism in the UCS testing methods can be divided into compression-shear,compression-tension,composite failure mode,and no obvious failure mode.The trends of these apparatuses are towards automation,digitization,precision,and multi-modal test.Two size correction methods are commonly used.One is to develop empirical correlation between the measured indices and the specimen size.The other is to use a standard specimen to calculate the size correction factor.Three to five input parameters are commonly utilized in soft computation models to predict the UCS of rocks.The selection of the test methods for the UCS measurement can be carried out according to the testing scenario and the specimen size.The engineers can gain a comprehensive understanding of the UCS testing methods and its potential developments in various rock engineering endeavors. 展开更多
关键词 Uniaxial compressive strength(UCS) UCS testing methods Test apparatus data processing
在线阅读 下载PDF
Multi-scale intelligent fusion and dynamic validation for high-resolution seismic data processing in drilling
5
作者 YUAN Sanyi XU Yanwu +2 位作者 XIE Renjun CHEN Shuai YUAN Junliang 《Petroleum Exploration and Development》 2025年第3期680-691,共12页
During drilling operations,the low resolution of seismic data often limits the accurate characterization of small-scale geological bodies near the borehole and ahead of the drill bit.This study investigates high-resol... During drilling operations,the low resolution of seismic data often limits the accurate characterization of small-scale geological bodies near the borehole and ahead of the drill bit.This study investigates high-resolution seismic data processing technologies and methods tailored for drilling scenarios.The high-resolution processing of seismic data is divided into three stages:pre-drilling processing,post-drilling correction,and while-drilling updating.By integrating seismic data from different stages,spatial ranges,and frequencies,together with information from drilled wells and while-drilling data,and applying artificial intelligence modeling techniques,a progressive high-resolution processing technology of seismic data based on multi-source information fusion is developed,which performs simple and efficient seismic information updates during drilling.Case studies show that,with the gradual integration of multi-source information,the resolution and accuracy of seismic data are significantly improved,and thin-bed weak reflections are more clearly imaged.The updated seismic information while-drilling demonstrates high value in predicting geological bodies ahead of the drill bit.Validation using logging,mud logging,and drilling engineering data ensures the fidelity of the processing results of high-resolution seismic data.This provides clearer and more accurate stratigraphic information for drilling operations,enhancing both drilling safety and efficiency. 展开更多
关键词 high-resolution seismic data processing while-drilling update while-drilling logging multi-source information fusion thin-bed weak reflection artificial intelligence modeling
在线阅读 下载PDF
An Advanced Image Processing Technique for Backscatter-Electron Data by Scanning Electron Microscopy for Microscale Rock Exploration 被引量:2
6
作者 Zhaoliang Hou Kunfeng Qiu +1 位作者 Tong Zhou Yiwei Cai 《Journal of Earth Science》 SCIE CAS CSCD 2024年第1期301-305,共5页
Backscatter electron analysis from scanning electron microscopes(BSE-SEM)produces high-resolution image data of both rock samples and thin-sections,showing detailed structural and geochemical(mineralogical)information... Backscatter electron analysis from scanning electron microscopes(BSE-SEM)produces high-resolution image data of both rock samples and thin-sections,showing detailed structural and geochemical(mineralogical)information.This allows an in-depth exploration of the rock microstructures and the coupled chemical characteristics in the BSE-SEM image to be made using image processing techniques.Although image processing is a powerful tool for revealing the more subtle data“hidden”in a picture,it is not a commonly employed method in geoscientific microstructural analysis.Here,we briefly introduce the general principles of image processing,and further discuss its application in studying rock microstructures using BSE-SEM image data. 展开更多
关键词 Image processing rock microstructures electron-based imaging data mining
原文传递
Big Data Application Simulation Platform Design for Onboard Distributed Processing of LEO Mega-Constellation Networks 被引量:1
7
作者 Zhang Zhikai Gu Shushi +1 位作者 Zhang Qinyu Xue Jiayin 《China Communications》 SCIE CSCD 2024年第7期334-345,共12页
Due to the restricted satellite payloads in LEO mega-constellation networks(LMCNs),remote sensing image analysis,online learning and other big data services desirably need onboard distributed processing(OBDP).In exist... Due to the restricted satellite payloads in LEO mega-constellation networks(LMCNs),remote sensing image analysis,online learning and other big data services desirably need onboard distributed processing(OBDP).In existing technologies,the efficiency of big data applications(BDAs)in distributed systems hinges on the stable-state and low-latency links between worker nodes.However,LMCNs with high-dynamic nodes and long-distance links can not provide the above conditions,which makes the performance of OBDP hard to be intuitively measured.To bridge this gap,a multidimensional simulation platform is indispensable that can simulate the network environment of LMCNs and put BDAs in it for performance testing.Using STK's APIs and parallel computing framework,we achieve real-time simulation for thousands of satellite nodes,which are mapped as application nodes through software defined network(SDN)and container technologies.We elaborate the architecture and mechanism of the simulation platform,and take the Starlink and Hadoop as realistic examples for simulations.The results indicate that LMCNs have dynamic end-to-end latency which fluctuates periodically with the constellation movement.Compared to ground data center networks(GDCNs),LMCNs deteriorate the computing and storage job throughput,which can be alleviated by the utilization of erasure codes and data flow scheduling of worker nodes. 展开更多
关键词 big data application Hadoop LEO mega-constellation multidimensional simulation onboard distributed processing
在线阅读 下载PDF
Data processing method for aerial testing of rotating accelerometer gravity gradiometer 被引量:1
8
作者 QIAN Xuewu TANG Hailiang 《中国惯性技术学报》 EI CSCD 北大核心 2024年第8期743-752,共10页
A novel method for noise removal from the rotating accelerometer gravity gradiometer(MAGG)is presented.It introduces a head-to-tail data expansion technique based on the zero-phase filtering principle.A scheme for det... A novel method for noise removal from the rotating accelerometer gravity gradiometer(MAGG)is presented.It introduces a head-to-tail data expansion technique based on the zero-phase filtering principle.A scheme for determining band-pass filter parameters based on signal-to-noise ratio gain,smoothness index,and cross-correlation coefficient is designed using the Chebyshev optimal consistent approximation theory.Additionally,a wavelet denoising evaluation function is constructed,with the dmey wavelet basis function identified as most effective for processing gravity gradient data.The results of hard-in-the-loop simulation and prototype experiments show that the proposed processing method has shown a 14%improvement in the measurement variance of gravity gradient signals,and the measurement accuracy has reached within 4E,compared to other commonly used methods,which verifies that the proposed method effectively removes noise from the gradient signals,improved gravity gradiometry accuracy,and has certain technical insights for high-precision airborne gravity gradiometry. 展开更多
关键词 airborne gravity gradiometer data processing band-passing filter evaluation function
在线阅读 下载PDF
High-speed electro-absorption modulated laser
9
作者 Zhenyao Li Chen Lyu +5 位作者 Xuliang Zhou Mengqi Wang Haotian Qiu Yejin Zhang Hongyan Yu Jiaoqing Pan 《Journal of Semiconductors》 2025年第11期7-18,共12页
Currently,the global 5G network,cloud computing,and data center industries are experiencing rapid development.The continuous growth of data center traffic has driven the vigorous progress in high-speed optical transce... Currently,the global 5G network,cloud computing,and data center industries are experiencing rapid development.The continuous growth of data center traffic has driven the vigorous progress in high-speed optical transceivers for optical interconnection within data centers.The electro-absorption modulated laser(EML),which is widely used in optical fiber communications,data centers,and high-speed data transmission systems,represents a high-performance photoelectric conversion device.Compared to traditional directly modulated lasers(DMLs),EMLs demonstrate lower frequency chirp and higher modulation bandwidth,enabling support for higher data rates and longer transmission distances.This article introduces the composition,working principles,manufacturing processes,and applications of EMLs.It reviews the progress on advanced indium phosphide(InP)-based EML devices from research institutions worldwide,while summarizing and comparing data transmission rates and key technical approaches across various studies. 展开更多
关键词 electro-absorption modulation high-speed laser modulation bandwidth data transmission rate
在线阅读 下载PDF
Hot deformation characteristics and microstructure evolution of industrial grade AISI M35 high-speed steel produced by ESR
10
作者 Wei Liang Jing Li +2 位作者 Jia-hao Li Xiao-yu Xiong Jian Chai 《Journal of Iron and Steel Research International》 2025年第8期2370-2388,共19页
The hot deformation behavior and microstructure evolution of industrial grade American Iron and Steel Institute(AISI)M35 high-speed steel produced by electroslag remelting at different parameters were investigated.The... The hot deformation behavior and microstructure evolution of industrial grade American Iron and Steel Institute(AISI)M35 high-speed steel produced by electroslag remelting at different parameters were investigated.The results indicated that grains coarsening and M2C carbides decomposing appeared in the steel at 1150℃for 5 min,and the network carbides were broken and deformed radially after the hot deformation.A constitutive equation was determined based on the corrected flow stress-strain curves considering the effects of friction and temperature,and a constitutive model with strain-compensated was established.The dynamic recrystallization(DRX)characteristic values were calculated based on the Cingara-McQueen model,and the grain distribution under different conditions was observed and analyzed.Significantly,the action mechanisms of carbides on the DRX were illuminated.It was found from a functional relation between average grain size and Z parameter that grain size increased with increasing temperature and decreasing strain rate.Optimal parameters for the hot deformation were determined as 980-1005℃~0.01-0.015 s^(−1)and 1095-1110℃~0.01-0.037 s^(−1)at the strain ranging from 0.05 to 0.8.Increasing the strain rate appropriately during deformation process was suggested to obtain fine and uniformly distributed carbides.Besides,an industrial grade forging deformation had also verified practicability of the above parameters. 展开更多
关键词 Electroslag remelted M35 high-speed steel Hot deformation CARBIDE Constitutive model processing map
原文传递
Chinese DeepSeek: Performance of Various Oversampling Techniques on Public Perceptions Using Natural Language Processing
11
作者 Anees Ara Muhammad Mujahid +2 位作者 Amal Al-Rasheed Shaha Al-Otaibi Tanzila Saba 《Computers, Materials & Continua》 2025年第8期2717-2731,共15页
DeepSeek Chinese artificial intelligence(AI)open-source model,has gained a lot of attention due to its economical training and efficient inference.DeepSeek,a model trained on large-scale reinforcement learning without... DeepSeek Chinese artificial intelligence(AI)open-source model,has gained a lot of attention due to its economical training and efficient inference.DeepSeek,a model trained on large-scale reinforcement learning without supervised fine-tuning as a preliminary step,demonstrates remarkable reasoning capabilities of performing a wide range of tasks.DeepSeek is a prominent AI-driven chatbot that assists individuals in learning and enhances responses by generating insightful solutions to inquiries.Users possess divergent viewpoints regarding advanced models like DeepSeek,posting both their merits and shortcomings across several social media platforms.This research presents a new framework for predicting public sentiment to evaluate perceptions of DeepSeek.To transform the unstructured data into a suitable manner,we initially collect DeepSeek-related tweets from Twitter and subsequently implement various preprocessing methods.Subsequently,we annotated the tweets utilizing the Valence Aware Dictionary and sentiment Reasoning(VADER)methodology and the lexicon-driven TextBlob.Next,we classified the attitudes obtained from the purified data utilizing the proposed hybrid model.The proposed hybrid model consists of long-term,shortterm memory(LSTM)and bidirectional gated recurrent units(BiGRU).To strengthen it,we include multi-head attention,regularizer activation,and dropout units to enhance performance.Topic modeling employing KMeans clustering and Latent Dirichlet Allocation(LDA),was utilized to analyze public behavior concerning DeepSeek.The perceptions demonstrate that 82.5%of the people are positive,15.2%negative,and 2.3%neutral using TextBlob,and 82.8%positive,16.1%negative,and 1.2%neutral using the VADER analysis.The slight difference in results ensures that both analyses concur with their overall perceptions and may have distinct views of language peculiarities.The results indicate that the proposed model surpassed previous state-of-the-art approaches. 展开更多
关键词 DeepSeek PREDICTION natural language processing deep learning analysis TextBlob imbalance data
在线阅读 下载PDF
Design of Differential Signal Processing Circuitry for Single-Frequency Laser Interferometry Displacement Measurement
12
作者 Songxiang Liu Jingping Yan Can Tang 《Journal of Electronic Research and Application》 2025年第2期258-267,共10页
This thesis addresses the issues existing in traditional laser tracking displacement measurement technology in the field of ultraprecision metrology by designing a differential signal processing circuit for high-preci... This thesis addresses the issues existing in traditional laser tracking displacement measurement technology in the field of ultraprecision metrology by designing a differential signal processing circuit for high-precision laser interferometric displacement measurement.A stable power supply module is designed to provide low-noise voltage to the entire circuit.An analog circuit system is constructed,including key circuits such as photoelectric sensors,I-V amplification,zero adjustment,fully differential amplification,and amplitude modulation filtering.To acquire and process signals,the PMAC Acc24E3 data acquisition card is selected,which realizes phase demodulation through reversible square wave counting,inverts displacement information,and a visual interface for the host computer is designed.Experimental verification shows that the designed system achieves micrometer-level measurement accuracy within a range of 0-10mm,with a maximum measurement error of less than 1.2μm,a maximum measurement speed of 6m/s,and a resolution better than 0.158μm. 展开更多
关键词 Displacement Measurement Weak Signal processing Differential Signal data Acquisition
在线阅读 下载PDF
Preliminary exploration of constructing a standardized process for prognostic biomarker discovery based on genetic big data
13
作者 Wang Min Yang Yongqi Li Xiawei 《China Standardization》 2025年第3期60-64,共5页
The paper utilized a standardized methodology to identify prognostic biomarkers in hepatocellular carcinoma(HCC)by analyzing transcriptomic and clinical data from The Cancer Genome Atlas(TCGA)database.The approach,whi... The paper utilized a standardized methodology to identify prognostic biomarkers in hepatocellular carcinoma(HCC)by analyzing transcriptomic and clinical data from The Cancer Genome Atlas(TCGA)database.The approach,which included stringent data preprocessing,differential gene expression analysis,and Kaplan-Meier survival analysis,provided valuable insights into the genetic underpinnings of HCC.The comprehensive analysis of a dataset involving 370 HCC patients uncovered correlations between survival status and pathological characteristics,including tumor size,lymph node involvement,and distant metastasis.The processed transcriptome dataset,comprising 420 samples and annotating 26,783 genes,served as a robust platform for identifying differential gene expression patterns.Among the significant differential expression genes,the key genes such as FBXO43,HAGLROS,CRISPLD1,LRRC3.DT,and ERN2,were pinpointed,which showed significant associations with patient survival outcomes,indicating their potential as novel prognostic biomarkers.This study can not only enhance the understanding of HCC’s genetic landscape but also establish a blueprint for a standardized process to discover prognostic biomarkers of various diseases using genetic big data.Future research should focus on validating these biomarkers through independent cohorts and exploring their utility in the development of personalized treatment strategies. 展开更多
关键词 standardized process genetic big data prognostic biomarkers Kaplan-Meier survival analysis hepatocellular carcinoma
暂未订购
The Mini-SiTian Array:Imaging Processing Pipeline
14
作者 Kai Xiao Zhirui Li +19 位作者 Yang Huang Jie Zheng Haibo Yuan Junju Du Linying Mi Hongrui Gu Yongkang Sun Bowen Zhang Shunxuan He Henggeng Han Min He Ruifeng Shi Yu Zhang Chuanjie Zheng Zexi Niu Guiting Tian Hu Zou Yongna Mao Hong Wu Jifeng Liu 《Research in Astronomy and Astrophysics》 2025年第4期55-70,共16页
As a pathfinder of the SiTian project,the Mini-SiTian(MST)Array,employed three commercial CMOS cameras,represents a next-generation,cost-effective optical time-domain survey project.This paper focuses primarily on the... As a pathfinder of the SiTian project,the Mini-SiTian(MST)Array,employed three commercial CMOS cameras,represents a next-generation,cost-effective optical time-domain survey project.This paper focuses primarily on the precise data processing pipeline designed for wide-field,CMOS-based devices,including the removal of instrumental effects,astrometry,photometry,and flux calibration.When applying this pipeline to approximately3000 observations taken in the Field 02(f02)region by MST,the results demonstrate a remarkable astrometric precision of approximately 70–80 mas(about 0.1 pixel),an impressive calibration accuracy of approximately1 mmag in the MST zero points,and a photometric accuracy of about 4 mmag for bright stars.Our studies demonstrate that MST CMOS can achieve photometric accuracy comparable to that of CCDs,highlighting the feasibility of large-scale CMOS-based optical time-domain surveys and their potential applications for cost optimization in future large-scale time-domain surveys,like the SiTian project. 展开更多
关键词 methods:data analysis techniques:image processing surveys
在线阅读 下载PDF
Cloud-Edge Collaborative Federated GAN Based Data Processing for IoT-Empowered Multi-Flow Integrated Energy Aggregation Dispatch
15
作者 Zhan Shi 《Computers, Materials & Continua》 SCIE EI 2024年第7期973-994,共22页
The convergence of Internet of Things(IoT),5G,and cloud collaboration offers tailored solutions to the rigorous demands of multi-flow integrated energy aggregation dispatch data processing.While generative adversarial... The convergence of Internet of Things(IoT),5G,and cloud collaboration offers tailored solutions to the rigorous demands of multi-flow integrated energy aggregation dispatch data processing.While generative adversarial networks(GANs)are instrumental in resource scheduling,their application in this domain is impeded by challenges such as convergence speed,inferior optimality searching capability,and the inability to learn from failed decision making feedbacks.Therefore,a cloud-edge collaborative federated GAN-based communication and computing resource scheduling algorithm with long-term constraint violation sensitiveness is proposed to address these challenges.The proposed algorithm facilitates real-time,energy-efficient data processing by optimizing transmission power control,data migration,and computing resource allocation.It employs federated learning for global parameter aggregation to enhance GAN parameter updating and dynamically adjusts GAN learning rates and global aggregation weights based on energy consumption constraint violations.Simulation results indicate that the proposed algorithm effectively reduces data processing latency,energy consumption,and convergence time. 展开更多
关键词 IOT federated learning generative adversarial network data processing multi-flowintegration energy aggregation dispatch
在线阅读 下载PDF
Machine Learning-based Identification of Contaminated Images in Light Curve Data Preprocessing
16
作者 Hui Li Rong-Wang Li +1 位作者 Peng Shu Yu-Qiang Li 《Research in Astronomy and Astrophysics》 SCIE CAS CSCD 2024年第4期287-295,共9页
Attitude is one of the crucial parameters for space objects and plays a vital role in collision prediction and debris removal.Analyzing light curves to determine attitude is the most commonly used method.In photometri... Attitude is one of the crucial parameters for space objects and plays a vital role in collision prediction and debris removal.Analyzing light curves to determine attitude is the most commonly used method.In photometric observations,outliers may exist in the obtained light curves due to various reasons.Therefore,preprocessing is required to remove these outliers to obtain high quality light curves.Through statistical analysis,the reasons leading to outliers can be categorized into two main types:first,the brightness of the object significantly increases due to the passage of a star nearby,referred to as“stellar contamination,”and second,the brightness markedly decreases due to cloudy cover,referred to as“cloudy contamination.”The traditional approach of manually inspecting images for contamination is time-consuming and labor-intensive.However,we propose the utilization of machine learning methods as a substitute.Convolutional Neural Networks and SVMs are employed to identify cases of stellar contamination and cloudy contamination,achieving F1 scores of 1.00 and 0.98 on a test set,respectively.We also explore other machine learning methods such as ResNet-18 and Light Gradient Boosting Machine,then conduct comparative analyses of the results. 展开更多
关键词 techniques:image processing methods:data analysis light pollution
在线阅读 下载PDF
Optimization study of station track utilization in high-speed railroad based on constraints of control in random origin and process
17
作者 Yajing Zheng Dekun Zhang 《Railway Sciences》 2024年第3期332-343,共12页
Purpose-The purpose of this paper is to eliminate the fluctuations in train arrival and departure times caused by skewed distributions in interval operation times.These fluctuations arise from random origin and proces... Purpose-The purpose of this paper is to eliminate the fluctuations in train arrival and departure times caused by skewed distributions in interval operation times.These fluctuations arise from random origin and process factors during interval operations and can accumulate over multiple intervals.The aim is to enhance the robustness of high-speed rail station arrival and departure track utilization schemes.Design/methodologylapproach-To achieve this objective,the paper simulates actual train operations,incorporating the fluctuations in interval operation times into the utilization of arrival and departure tracks at the station.The Monte Carlo simulation method is adopted to solve this problem.This approach transforms a nonlinear model,which includes constraints from probability distribution functions and is difficult to solve directly,into a linear programming model that is easier to handle.The method then linearly weights two objectives to optimize the solution.Findings-Through the application of Monte Carlo simulation,the study successfully converts the complex nonlinear model with probability distribution function constraints into a manageable linear programming model.By continuously adjusting the weighting coefficients of the linear objectives,the method is able to optimize the Pareto solution.Notably,this approach does not require extensive scene data to obtain a satisfactory Pareto solution set.Originality/value-The paper contributes to the field by introducing a novel method for optimizing high-speed rail station arrival and departure track utilization in the presence of fluctuations in interval operation times.The use of Monte Carlo simulation to transform the problem into a tractable linear programming model represents a significant advancement.Furthermore,the method's ability to produce satisfactory Pareto solutions without relying on extensive data sets adds to its practical value and applicability in real-world scenarios. 展开更多
关键词 Control in random origin Control in random process high-speed railroad station Arrival and departure track utilization Optimization Paper type Research paper
在线阅读 下载PDF
Optimizing Healthcare Big Data Processing with Containerized PySpark and Parallel Computing: A Study on ETL Pipeline Efficiency
18
作者 Ehsan Soltanmohammadi Neset Hikmet 《Journal of Data Analysis and Information Processing》 2024年第4期544-565,共22页
In this study, we delve into the realm of efficient Big Data Engineering and Extract, Transform, Load (ETL) processes within the healthcare sector, leveraging the robust foundation provided by the MIMIC-III Clinical D... In this study, we delve into the realm of efficient Big Data Engineering and Extract, Transform, Load (ETL) processes within the healthcare sector, leveraging the robust foundation provided by the MIMIC-III Clinical Database. Our investigation entails a comprehensive exploration of various methodologies aimed at enhancing the efficiency of ETL processes, with a primary emphasis on optimizing time and resource utilization. Through meticulous experimentation utilizing a representative dataset, we shed light on the advantages associated with the incorporation of PySpark and Docker containerized applications. Our research illuminates significant advancements in time efficiency, process streamlining, and resource optimization attained through the utilization of PySpark for distributed computing within Big Data Engineering workflows. Additionally, we underscore the strategic integration of Docker containers, delineating their pivotal role in augmenting scalability and reproducibility within the ETL pipeline. This paper encapsulates the pivotal insights gleaned from our experimental journey, accentuating the practical implications and benefits entailed in the adoption of PySpark and Docker. By streamlining Big Data Engineering and ETL processes in the context of clinical big data, our study contributes to the ongoing discourse on optimizing data processing efficiency in healthcare applications. The source code is available on request. 展开更多
关键词 Big data Engineering ETL Healthcare Sector Containerized Applications Distributed Computing Resource Optimization data processing Efficiency
在线阅读 下载PDF
Data Analysis Methods and Signal Processing Techniques in Gravitational Wave Detection
19
作者 Bojun Yan 《Journal of Applied Mathematics and Physics》 2024年第11期3774-3783,共10页
Gravitational wave detection is one of the most cutting-edge research areas in modern physics, with its success relying on advanced data analysis and signal processing techniques. This study provides a comprehensive r... Gravitational wave detection is one of the most cutting-edge research areas in modern physics, with its success relying on advanced data analysis and signal processing techniques. This study provides a comprehensive review of data analysis methods and signal processing techniques in gravitational wave detection. The research begins by introducing the characteristics of gravitational wave signals and the challenges faced in their detection, such as extremely low signal-to-noise ratios and complex noise backgrounds. It then systematically analyzes the application of time-frequency analysis methods in extracting transient gravitational wave signals, including wavelet transforms and Hilbert-Huang transforms. The study focuses on discussing the crucial role of matched filtering techniques in improving signal detection sensitivity and explores strategies for template bank optimization. Additionally, the research evaluates the potential of machine learning algorithms, especially deep learning networks, in rapidly identifying and classifying gravitational wave events. The study also analyzes the application of Bayesian inference methods in parameter estimation and model selection, as well as their advantages in handling uncertainties. However, the research also points out the challenges faced by current technologies, such as dealing with non-Gaussian noise and improving computational efficiency. To address these issues, the study proposes a hybrid analysis framework combining physical models and data-driven methods. Finally, the research looks ahead to the potential applications of quantum computing in future gravitational wave data analysis. This study provides a comprehensive theoretical foundation for the optimization and innovation of gravitational wave data analysis methods, contributing to the advancement of gravitational wave astronomy. 展开更多
关键词 Gravitational Wave Detection data Analysis Signal processing Matched Filtering Machine Learning
在线阅读 下载PDF
HIGH-SPEED SPECTRAL DOMAIN OPTICAL COHERENCE TOMOGRAPHY SIGNAL PROCESSING WITH TIME-DOMAIN INTERPOLATION USING GRAPHICS PROCESSING UNIT
20
作者 XIQI LI GUOHUA SHI +2 位作者 LING WEI ZHIHUA DING YUDONG ZHANG 《Journal of Innovative Optical Health Sciences》 SCIE EI CAS 2011年第3期325-335,共11页
Sensitivity and data processing speed are important in spectral domain Optical Coherence Tomography(SD-OCT)system.To get a higher sensitivity,zero-padding interpolation together with linear interpolation is commonly u... Sensitivity and data processing speed are important in spectral domain Optical Coherence Tomography(SD-OCT)system.To get a higher sensitivity,zero-padding interpolation together with linear interpolation is commonly used to re-sample the interference data in SD-OCT,which limits the data processing speed.Recently,a time-domain interpolation for SD-OCT was proposed.By eliminating the huge Fast Fourier Transform Algorithm(FFT)operations,the operation number of the time-domain interpolation is much less than that of the zero-padding interpolation.In this paper,a numerical simulation is performed to evaluate the computational complexity and the interpolation accuracy.More than six times acceleration is obtained.At the same time,the normalized mean square error(NMSE)results show that the time-domain interpolation method with cut-offlength L?21 and L?31 can improve about 1.7 dB and 2.1 dB when the distance mismatch is 2.4mm than that of zero-padding interpolation method with padding times M?4,respectively.Furthermore,this method can be applied the parallel arithmetic processing because only the data in the cut-offwindow is processed.By using Graphics Processing Unit(GPU)with compute unified device architecture(CUDA)program model,a frame(400 A-lines2048 pixels12 bits)data can be processed in 6 ms and the processing capability can be achieved 164,000 line/s for 1024-OCT and 71,000 line/s for 2048-OCT when the cut-offlength is 21.Thus,a high-sensitivity and ultra-high data processing SD-OCT is realized. 展开更多
关键词 SD-DCT time-domain interpolation GPU CUDA data processing
原文传递
上一页 1 2 250 下一页 到第
使用帮助 返回顶部