摘要
由于异常数据的出现会导致统计分析发生显著变异,使得样本对总体的推断、控制与预测等工作产生不准确或者出现错误,因此有效的检测、追踪和防治异常具有重要的意义。首先提出异常数据的概念与定义,并且给出以"对象与属性"的描述方式定义了数据存在的结构形式。然后提出并分析关于理论分布与属性数据分布之间的相似度概念,建立属性数据分布的相似性求解算法。通过求得的相似分布和文中定义的可接受频率值,设定自适应门限阈值,最终建立以概率分布发现与识别单属性异常数据的算法。通过算法计算提高了数据的合理性与有效性,为数据挖掘处理提供了良好的数据环境。
Since the appearance of abnormal data may lead to remarkable variation in statistical analysis,which makes the samples inaccurate or wrong on the works of overall judgment,control and prediction,etc.,therefore effective detection,tracking and prevention and control of abnormal data have the significant importance.In this paper,first the concept and definition of abnormal data are presented,and the ＂object-property＂ description way is given to define the structure form of data existence.Then the similarity concept between theory distribution and attribute data distribution is proposed and analysed,and the similarity solution algorithm of attribute data distribution is established as well.Finally,through the similarity distribution obtained and the acceptable frequency value defined in the paper and the setting of the self-adaptive threshold,an algorithm of detection and recognition of abnormal data with single attribute by probability distribution is established,which not only improves the rationality and effectiveness of data,but also provides a good data environment for data mining processing.
出处
《计算机应用与软件》
CSCD
北大核心
2012年第11期139-140,164,共3页
Computer Applications and Software
基金
全军军事学研究生课题(2010JY0585-335)
