摘要
大气污染条件下气象监测数据种类较多、特征混乱导致提取结果复杂,为此,提出基于大数据挖掘的数据采集处理方法。考虑到气象监测数据中包含的风向、风速、温度、湿度、降雨量等多个尺度的参数,利用聚类算法对海量的气象监测数据进行分组,识别出数据中的自然类别和模式。通过计算各类别的中心值并应用聚类收敛规则,计算不同区域间的样本相似度,通过相似性函数确定聚类采集数学式。利用线性分析算法得出聚类采集数据集中属性最大值和最小值,设定映射曲线实现线性映射,规范数据属性。计算规范化数据的时间序列均值后再计算中值,并按照多尺度参数依次求解,根据标准差值实现校对滤波,明确不同气象条件下的数据特征。实验数据证明,所提方法对气象监测数据采集处理效果较好,有效改善了初始序列中噪点和毛刺问题。
The meteorological monitoring data under the condition of atmospheric pollution have more types and confusing features that lead to complicated extraction results;therefore,a data collection and processing method based on big data mining is proposed.Considering the wind direction,wind speed,temperature,humidity,rainfall and other scale parameters contained in the meteorological monitoring data,a clustering algorithm is used to group the huge amount of meteorological monitoring data and identify the natural categories and patterns in the data.By calculating the center value of each category and applying the convergence rule of clustering,the sample similarity between different regions is calculated,and the mathematical formula of cluster collection is determined by the similarity function.Using a linear analysis algorithm to derive the maximum and minimum values of the attributes inthe clustered collection data set,set the mapping curve to realize linear mapping and normalize the data attributes.After calculating the mean value of the normalized data,the median value is calculated,and the data are solved sequentially according to the multi-scale parameters,and the standard deviation value is used to realize the calibration filtering,so as to clarify the characteristics of the data under different meteorological conditions.The experimental data proved that the proposed method is effective in meteorological monitoring data acquisition and processing,and effectively improves the noise and burr problems in the initial series.
作者
刘艳群
陈创买
黄观荣
王敏
LIU Yan-qun;CHEN Chuang-mai;HUANG Guan-rong;WANG Min(Bureau of Meteorology of Shaoguan City,Guangdong province,Guangdong Shaoguan 512028,China;School of Atmosphere Sciences,Sun Yat-sen University,Guangdong Guangzhou 510275,China)
出处
《计算机仿真》
2025年第11期315-319,共5页
Computer Simulation
基金
韶关市科学技术局项目(230616148033947)
广东省气象局面上项目(GRMC2023M45)。
关键词
大数据挖掘
大气污染
气象监测
数据采集处理
相似性函数
线性映射
Big data mining
Air pollution
Meteorological monitoring
Data acquisition and processing
Similarity functions
Linear mapping
