期刊文献+
共找到1,421篇文章
< 1 2 72 >
每页显示 20 50 100
Enhancing IoT Resilience at the Edge:A Resource-Efficient Framework for Real-Time Anomaly Detection in Streaming Data
1
作者 Kirubavathi G. Arjun Pulliyasseri +5 位作者 Aswathi Rajesh Amal Ajayan Sultan Alfarhood Mejdl Safran Meshal Alfarhood Jungpil Shin 《Computer Modeling in Engineering & Sciences》 2025年第6期3005-3031,共27页
The exponential expansion of the Internet of Things(IoT),Industrial Internet of Things(IIoT),and Transportation Management of Things(TMoT)produces vast amounts of real-time streaming data.Ensuring system dependability... The exponential expansion of the Internet of Things(IoT),Industrial Internet of Things(IIoT),and Transportation Management of Things(TMoT)produces vast amounts of real-time streaming data.Ensuring system dependability,operational efficiency,and security depends on the identification of anomalies in these dynamic and resource-constrained systems.Due to their high computational requirements and inability to efficiently process continuous data streams,traditional anomaly detection techniques often fail in IoT systems.This work presents a resource-efficient adaptive anomaly detection model for real-time streaming data in IoT systems.Extensive experiments were carried out on multiple real-world datasets,achieving an average accuracy score of 96.06%with an execution time close to 7.5 milliseconds for each individual streaming data point,demonstrating its potential for real-time,resourceconstrained applications.The model uses Principal Component Analysis(PCA)for dimensionality reduction and a Z-score technique for anomaly detection.It maintains a low computational footprint with a sliding window mechanism,enabling incremental data processing and identification of both transient and sustained anomalies without storing historical data.The system uses a Multivariate Linear Regression(MLR)based imputation technique that estimates missing or corrupted sensor values,preserving data integrity prior to anomaly detection.The suggested solution is appropriate for many uses in smart cities,industrial automation,environmental monitoring,IoT security,and intelligent transportation systems,and is particularly well-suited for resource-constrained edge devices. 展开更多
关键词 Anomaly detection streaming data IOT IIoT TMoT REAL-TIME LIGHTWEIGHT modeling
在线阅读 下载PDF
Modeling and Performance Evaluation of Streaming Data Processing System in IoT Architecture
2
作者 Feng Zhu Kailin Wu Jie Ding 《Computers, Materials & Continua》 2025年第5期2573-2598,共26页
With the widespread application of Internet of Things(IoT)technology,the processing of massive realtime streaming data poses significant challenges to the computational and data-processing capabilities of systems.Alth... With the widespread application of Internet of Things(IoT)technology,the processing of massive realtime streaming data poses significant challenges to the computational and data-processing capabilities of systems.Although distributed streaming data processing frameworks such asApache Flink andApache Spark Streaming provide solutions,meeting stringent response time requirements while ensuring high throughput and resource utilization remains an urgent problem.To address this,the study proposes a formal modeling approach based on Performance Evaluation Process Algebra(PEPA),which abstracts the core components and interactions of cloud-based distributed streaming data processing systems.Additionally,a generic service flow generation algorithmis introduced,enabling the automatic extraction of service flows fromthe PEPAmodel and the computation of key performance metrics,including response time,throughput,and resource utilization.The novelty of this work lies in the integration of PEPA-based formal modeling with the service flow generation algorithm,bridging the gap between formal modeling and practical performance evaluation for IoT systems.Simulation experiments demonstrate that optimizing the execution efficiency of components can significantly improve system performance.For instance,increasing the task execution rate from 10 to 100 improves system performance by 9.53%,while further increasing it to 200 results in a 21.58%improvement.However,diminishing returns are observed when the execution rate reaches 500,with only a 0.42%gain.Similarly,increasing the number of TaskManagers from 10 to 20 improves response time by 18.49%,but the improvement slows to 6.06% when increasing from 20 to 50,highlighting the importance of co-optimizing component efficiency and resource management to achieve substantial performance gains.This study provides a systematic framework for analyzing and optimizing the performance of IoT systems for large-scale real-time streaming data processing.The proposed approach not only identifies performance bottlenecks but also offers insights into improving system efficiency under different configurations and workloads. 展开更多
关键词 System modeling performance evaluation streaming data process IoT system PEPA
在线阅读 下载PDF
Environment-aware streaming media transmission method in high-speed mobile networks
3
作者 Jia Guo Jinqi Zhu +3 位作者 Xiang Li Bowen Sun Qian Gao Weijia Feng 《Digital Communications and Networks》 2025年第4期991-1005,共15页
With technological advancements,high-speed rail has emerged as a prevalent mode of transportation.During travel,passengers exhibit a growing demand for streaming media services.However,the high-speed mobile networks e... With technological advancements,high-speed rail has emerged as a prevalent mode of transportation.During travel,passengers exhibit a growing demand for streaming media services.However,the high-speed mobile networks environment poses challenges,including frequent base station handoffs,which significantly degrade wireless network transmission performance.Improving transmission efficiency in high-speed mobile networks and optimizing spatiotemporal wireless resource allocation to enhance passengers’media experiences are key research priorities.To address these issues,we propose an Adaptive Cross-Layer Optimization Transmission Method with Environment Awareness(ACOTM-EA)tailored for high-speed rail streaming media.Within this framework,we develop a channel quality prediction model utilizing Kalman filtering and an algorithm to identify packet loss causes.Additionally,we introduce a proactive base station handoffstrategy to minimize handoffrelated disruptions and optimize resource distribution across adjacent base stations.Moreover,this study presents a wireless resource allocation approach based on an enhanced genetic algorithm,coupled with an adaptive bitrate selection mechanism,to maximize passenger Quality of Experience(QoE).To evaluate the proposed method,we designed a simulation experiment and compared ACOTM-EA with established algorithms.Results indicate that ACOTM-EA improves throughput by 11%and enhances passengers’media experience by 5%. 展开更多
关键词 High-speed mobile networks streaming media Environment-aware Kalman filtering Resource allocation
在线阅读 下载PDF
Ultrafast no-wash bioassay based on gold nanoparticles and enhanced by acoustic streaming
4
作者 Shuting Pan Xianwu Ke +7 位作者 Rui You Yanyan Wang Xian Chen Xiaotian Shen Bohua Liu Chongling Sun Wei Pang Xuexin Duan 《Nanotechnology and Precision Engineering》 2025年第2期77-85,共9页
No-wash bioassays based on nanoparticles are used widely in biochemical procedures because of their responsive sensing and no need forwashing processes.Essential for no-wash biosensing are the interactions between nan... No-wash bioassays based on nanoparticles are used widely in biochemical procedures because of their responsive sensing and no need forwashing processes.Essential for no-wash biosensing are the interactions between nanoparticles and biomolecules,but it is challenging toachieve controlled bioconjugation of molecules on nanomaterials.Reported here is a way to actively improve nanoparticle-based no-washbioassays by enhancing the binding between biomolecules and gold nanoparticles via acoustic streaming generated by a gigahertz piezoelectricnanoelectromechanical resonator.Tunable micro-vortices are generated at the device-liquid interface,thereby accelerating the internalcirculating flow of the solution,bypassing the diffusion limitation,and thus improving the binding between the biomolecules and goldnanoparticles.Combined with fluorescence quenching,an enhanced and ultrafast no-wash biosensing assay is realized for specific proteins.The sensing method presented here is a versatile tool for different types of biomolecule detection with high efficiency and simplicity. 展开更多
关键词 Acoustic streaming Gold nanoparticles Fluorescence quenching BIOSENSING
在线阅读 下载PDF
基于嵌入式Linux系统的Smooth Streaming流媒体设计与实现
5
作者 刘刚 《信息与电脑》 2025年第12期139-142,共4页
随着视频播放需求的爆发性增长,在嵌入式硬件资源受限的情况下,如何保障视频流畅播放成为亟待解决的问题。文章设计并实现了一种基于嵌入式Linux系统的Smooth Streaming流媒体解决方案。基于微软公司Smooth Streaming流媒体技术的机制,... 随着视频播放需求的爆发性增长,在嵌入式硬件资源受限的情况下,如何保障视频流畅播放成为亟待解决的问题。文章设计并实现了一种基于嵌入式Linux系统的Smooth Streaming流媒体解决方案。基于微软公司Smooth Streaming流媒体技术的机制,提出分层设计方案,包含流媒体解析模块、流媒体解密模块、流媒体下载模块和流媒体动态码率控制模块。方案在嵌入式Linux平台完成代码开发与系统测试,已成功应用于商用产品。测试结果表明,在硬件资源受限条件下,该方案可实现视频的流畅播放,且系统可长期稳定运行。 展开更多
关键词 嵌入式LINUX Smooth streaming 流媒体
在线阅读 下载PDF
A Review of Content Marketing’s Influence on Consumers’ Purchase Intention in Live-streaming E-commerce
6
作者 Yuan Wang 《Proceedings of Business and Economic Studies》 2025年第1期55-59,共5页
In the contemporary digital landscape,the proliferation of information has led to an increasing diversity of channels through which consumers obtain information,resulting in a gradual transformation of shopping habits... In the contemporary digital landscape,the proliferation of information has led to an increasing diversity of channels through which consumers obtain information,resulting in a gradual transformation of shopping habits.Consumers now frequently rely on external sources to make well-informed purchasing decisions,leading to the emergence of live shopping as a prominent avenue for gathering product information and completing transactions.E-commerce live streaming has experienced rapid growth,leveraging its ability to generate traffic and capture consumer attention.The integration of content and live streaming not only meets users’psychological needs but also facilitates seamless communication between buyers and sellers.From the perspective of content marketing typologies,this paper examines content marketing across three key dimensions:informational content,entertainment content,and emotional content.It further explores the impact of content marketing on consumers’purchase intentions within the context of e-commerce live streaming. 展开更多
关键词 Content marketing E-commerce live streaming Consumer purchase intention
在线阅读 下载PDF
Secure task offloading strategy optimization of UAV-aided outdoor mobile high-definition live streaming
7
作者 Ming YAN Yuxuan ZHANG +2 位作者 Chien Aun CHAN André F.GYGAX Chunguo LI 《Chinese Journal of Aeronautics》 2025年第10期201-215,共15页
Unmanned aerial vehicles(UAVs)bring more innovation and attraction to outdoor mobile high-definition(HD)live streaming with its unique perspective.Due to the heavy computational requirements of HD live broadcast tasks... Unmanned aerial vehicles(UAVs)bring more innovation and attraction to outdoor mobile high-definition(HD)live streaming with its unique perspective.Due to the heavy computational requirements of HD live broadcast tasks and the limited hardware performance of UAV equipment,how to reduce the system response delay and improve the energy efficiency of terminal equipment directly affects the secure broadcast of the system.Secure task offloading in this scenario is considered a promising solution and has received academic attention.In this paper,we simulate the UAV-aided outdoor mobile HD live streaming scenarios and optimize the relevant task offloading strategies.First,we design the total cost function of task offloading that jointly optimizes secure time latency and energy consumption.Additionally,we propose a collaborative computing model for multi-UAV task offloading.This model combines the idea of simulated annealing(SA)and introduces the compression factor to enhance the particle swarm optimization(PSO)to realize secure task offloading.The simulation results show that the proposed strategy has better performance in balancing network latency and energy consumption.Compared with the discrete teaching–learning-based optimization(DTLBO)and quantum PSO(QPSO)task offloading strategies,the fitness value of the proposed strategy is decreased by an average of 26.73%and 16.42%,respectively. 展开更多
关键词 Unmanned aerial vehicle(UAV) High-definition(HD)live streaming Secure task offloading Network energy consumption Particle swarm optimization(PSO)
原文传递
Design and Implementation of a Proxy Caching System for Streaming Media
8
作者 TanJin YuSheng-shengI ZhouJing-li 《Wuhan University Journal of Natural Sciences》 EI CAS 2004年第1期31-36,共6页
With the widespread use of streaming media application on the Internet, a significant change in Internet workload will be provoked. Caching is one of the applied techniques for enhancing the scalability of streaming s... With the widespread use of streaming media application on the Internet, a significant change in Internet workload will be provoked. Caching is one of the applied techniques for enhancing the scalability of streaming system and reducing the workload of server/network. Aiming at the characteristics of broadband network in community, we propose a popularity-based server-proxy caching strategy for streaming medias, and implement the prototype of streaming proxy caching based on this strategy, using RTSP as control protocol and RTP for content transport. This system can play a role in decreasing server load, reducing the traffic from streaming server to proxy, and improving the start-up latency of the client. Key words streaming server - proxy - cache - streaming media - real time streaming protocol CLC number TP 302 - TP 333 Foundation item: Supported by the National High Technology Development 863 Program of China (2001AA111011).Biography: Tan Jin (1962-), male, Ph. D candidate, research direction: network communications, multimedia technologies, and web caching. 展开更多
关键词 streaming server PROXY CACHE streaming media real time streaming protocol
在线阅读 下载PDF
基于Spark Streaming的实时数据分析系统及其应用 被引量:30
9
作者 韩德志 陈旭光 +2 位作者 雷雨馨 戴永涛 张肖 《计算机应用》 CSCD 北大核心 2017年第5期1263-1269,共7页
为了实现对实时网络数据流的快速分析,设计一种分布式实时数据流分析系统(DRDAS),能有效解决并发访问数据流的收集、存储和实时分析问题,为大数据环境的网络安全检测提供了一种有效的数据分析平台;根据Spark Streaming运行的原理设计一... 为了实现对实时网络数据流的快速分析,设计一种分布式实时数据流分析系统(DRDAS),能有效解决并发访问数据流的收集、存储和实时分析问题,为大数据环境的网络安全检测提供了一种有效的数据分析平台;根据Spark Streaming运行的原理设计一种动态采样的K-Means并行算法,与DRDAS结合能实时有效地检测大数据环境下的各种分布式拒绝服务(DDo S)攻击。实验结果显示:DRDAS具有好的可扩展性、容错性和实时处理能力,与动态采样的K-Means并行算法结合能实时地检测各种DDo S攻击,缩短了攻击的检测时间。 展开更多
关键词 SPARK streaming框架 分布式流处理 网络数据分析 分布式拒绝服务攻击
在线阅读 下载PDF
面向直播HTTP Streaming系统的HTTP缓存服务器行为优化 被引量:7
10
作者 李云飞 谢伟凯 +2 位作者 鲁晨平 张智强 申瑞民 《计算机工程与应用》 CSCD 2012年第10期68-74,共7页
HTTP缓存服务器是提高HTTP Streaming系统客户并发量的关键环节。但当前主流HTTP缓存服务器,如Nginx、Squid、Varnish等,在缓存资源更新期间的行为都存在不足,当被应用在面向直播的HTTP Streaming系统中时,会周期性地把大量客户端请求... HTTP缓存服务器是提高HTTP Streaming系统客户并发量的关键环节。但当前主流HTTP缓存服务器,如Nginx、Squid、Varnish等,在缓存资源更新期间的行为都存在不足,当被应用在面向直播的HTTP Streaming系统中时,会周期性地把大量客户端请求转发至源服务器,从而制约了HTTP Streaming系统的可伸缩性。提出一种优化的HTTP缓存服务器在缓存更新期间的行为,即缓存服务器仅向源服务器转发一路客户端请求,缓存更新期间,拒绝其他关于该资源的请求。优化策略在使用最为广泛的Nginx服务器的基础上进行了实现。实验证明,优化后系统的伸缩性得到了显著提高。 展开更多
关键词 HTTP streaming 缓存服务器 缓存更新
在线阅读 下载PDF
使用Spark Streaming的自适应实时DDoS检测和防御技术 被引量:10
11
作者 方峰 蔡志平 +2 位作者 肇启佳 林加润 朱明 《计算机科学与探索》 CSCD 北大核心 2016年第5期601-611,共11页
分布式拒绝服务(distributed denial of service,DDoS)攻击是重要的安全威胁,网络速度的不断提高给传统的检测方法带来了新的挑战。以Spark等为代表的大数据处理技术,给网络安全的高速检测带来了新的契机。提出了一种基于Spark Streamin... 分布式拒绝服务(distributed denial of service,DDoS)攻击是重要的安全威胁,网络速度的不断提高给传统的检测方法带来了新的挑战。以Spark等为代表的大数据处理技术,给网络安全的高速检测带来了新的契机。提出了一种基于Spark Streaming框架的自适应实时DDoS检测防御技术,通过对滑动窗口内源簇进行分组,并根据与各分组内源簇比例的偏差统计,检测出DDoS攻击流量。通过感知合法的网络流量,实现了对DDoS攻击的自适应快速检测和有效响应。实验结果表明,该技术可极大地提升检测能力,为保障网络服务性能和安全检测的可扩展性提供了一种可行的解决方案。 展开更多
关键词 DDOS检测 DDOS防御 实时检测 自适应检测 SPARK streaming
在线阅读 下载PDF
基于Spark Streaming流回归的煤矿瓦斯浓度实时预测 被引量:11
12
作者 吴海波 施式亮 念其锋 《中国安全生产科学技术》 CAS CSCD 北大核心 2017年第5期84-89,共6页
为了实时分析瓦斯监测流数据并对瓦斯浓度进行准确预测以实现瓦斯灾害实时预警,以实时流数据处理框架Spark Streaming构建基于流回归的瓦斯浓度实时预测系统。系统采用分布式流处理技术,可使基于回归算法的瓦斯浓度预测模型更新周期达... 为了实时分析瓦斯监测流数据并对瓦斯浓度进行准确预测以实现瓦斯灾害实时预警,以实时流数据处理框架Spark Streaming构建基于流回归的瓦斯浓度实时预测系统。系统采用分布式流处理技术,可使基于回归算法的瓦斯浓度预测模型更新周期达到秒级,提高了瓦斯浓度预测精度,满足流式大数据处理的实时性要求。实验表明:应用Spark Streaming流回归预测系统在采样周期为5s的瓦斯监测数据流上进行实时预测时,预测平均均方根误差随模型更新周期的缩短而减小,模型更新周期可达15s,且更新周期为45s时预测总均方根误差最小,既能保证预测精度,又能提高瓦斯灾害预警时效。 展开更多
关键词 监测数据 流数据 瓦斯浓度 SPARK streaming 流回归 实时预测 灾害预警
在线阅读 下载PDF
基于Spark Streaming的实时能耗分项计量系统 被引量:9
13
作者 武志学 《计算机应用》 CSCD 北大核心 2017年第4期928-935,共8页
能耗分项计量能够准确、及时、有效地发现能源使用问题,形成和实现最有效的节能措施。能耗分项计量系统需要对各项能源使用量在不同粒度上进行统计,既有实时性的需求,又需要涉及到聚合、去重、连接等较为复杂的统计需求。由于数据产生... 能耗分项计量能够准确、及时、有效地发现能源使用问题,形成和实现最有效的节能措施。能耗分项计量系统需要对各项能源使用量在不同粒度上进行统计,既有实时性的需求,又需要涉及到聚合、去重、连接等较为复杂的统计需求。由于数据产生快、实时性强、数据量大,所以很难统一采集并入库存储后再作处理,这便导致传统的数据处理架构不能满足需求。为此,提出基于Spark Streaming大数据流式技术构建一个实时能耗分项计量系统,对实时能耗分项计量的系统架构和内部结构进行了详细介绍,并通过实验数据分析了系统的实时数据处理能力。与传统架构不同,实时能耗分项计量系统在数据流动的过程中实时地进行捕捉和处理,一方面把捕捉到的异常信息及时报警到前端,同时把分类分项统计处理的结果保存到数据库,以便进行离线分析和数据挖掘,能有效地解决上述数据处理过程中遇到的问题。 展开更多
关键词 流式计算 能耗分项计量 SPARK streaming APACHE Kafka 大数据
在线阅读 下载PDF
基于Spark Streaming的视频/图像流处理与新的性能评估方法 被引量:10
14
作者 黄文辉 冯瑞 《计算机工程与科学》 CSCD 北大核心 2015年第11期2055-2060,共6页
智能视频监控技术在公共安全、交通管理、智慧城市等方面有着广泛的运用前景,需求日益增长。随着摄像头安装的数量越来越多,采集的图像数据量越来越大,靠单台计算机处理已经远远不能满足需求了。分布式计算的兴起与发展为解决大规模的... 智能视频监控技术在公共安全、交通管理、智慧城市等方面有着广泛的运用前景,需求日益增长。随着摄像头安装的数量越来越多,采集的图像数据量越来越大,靠单台计算机处理已经远远不能满足需求了。分布式计算的兴起与发展为解决大规模的数据处理问题提供了很好的途径。使用一种基于Spark Streaming的视频/图像流处理的测试平台,阐述了平台的构成和工作流程,深入研究各个参数对集群性能的影响,创新性地提出了CPU时间占用率作为性能评估指标,与总的处理时间结合,更为全面反映集群性能和资源利用率。 展开更多
关键词 SPARK streaming CPU时间占用率 分布式 视频/图像处理
在线阅读 下载PDF
SDHT: Efficient One-Hop DHT Lookup Framework for P2P Live Streaming 被引量:2
15
作者 Chen Zhuo Feng Gang +1 位作者 Zhou Yang Lu Yi 《China Communications》 SCIE CSCD 2012年第8期88-104,共17页
Video data location plays a key role for Peer-to-Peer (P2P) live streaming applications. In this paper, we propose a new one-hop Distributed Hash Table (DHT) lookup frarrework called Strearre ing-DHT (SDHT) to p... Video data location plays a key role for Peer-to-Peer (P2P) live streaming applications. In this paper, we propose a new one-hop Distributed Hash Table (DHT) lookup frarrework called Strearre ing-DHT (SDHT) to provide efficient video data location service. By adopting an enhanced events dissemination mechanism-EDRA+, the accuracy of routing table at peers can be guaranteed. More importantly, in order to enhance the perforlmnce of video data lookup operation without incurring extra overhead, we design a so-called Distributed Index Mapping and Management Mechanism (DIMM) for SDHT. Both analytical modeling and intensive simulation experiments are conducted to demonstrate the effectiveness of SDHT framework. Numerical results show that almost 90% requested video data can be retrieved within one second in SDHT based systems, and SDHT needs only 26% average bandwidth consumption when compared with similar one-hop DHT solutions such as D1HT. This indicates that SDHT framework is an appropriate data lookup solution for time-sensitive network applications such as P2P live streaming. 展开更多
关键词 PEER-TO-PEER distributed hash table streaming application live streaming EDRA
在线阅读 下载PDF
基于Hadoop Streaming的Last比对软件并行化的研究与实现
16
作者 董本志 李文浩 景维鹏 《计算机工程与应用》 CSCD 2014年第2期226-230,共5页
随着下一代测序技术的到来,单机版Last比对软件已经不能满足海量数据的处理需求。使用Hadoop Streaming技术将Last比对软件快速部署到云计算环境中,解决当前单机版Last比对软件处理大数据能力差的问题。通过自定义的基于NFS文件系统的... 随着下一代测序技术的到来,单机版Last比对软件已经不能满足海量数据的处理需求。使用Hadoop Streaming技术将Last比对软件快速部署到云计算环境中,解决当前单机版Last比对软件处理大数据能力差的问题。通过自定义的基于NFS文件系统的数据集切分方法和基于Partitioner的任务分配方式能够实现均衡高效的数据切分,并保证并行化粒度可控。实验结果表明,在保证与单机运行结果一致的情况下,这种方法能有效缩减软件运行时间,具有较高的加速比。 展开更多
关键词 HADOOP streaming 软件并行化 last比对软件
在线阅读 下载PDF
医学影像系统Streaming框架的设计和实现
17
作者 赵晨晖 谢顺华 +1 位作者 吕旭东 段会龙 《中国医疗器械杂志》 CAS 2004年第6期425-426,419,共3页
提出医学影像Streaming概念,并设计和实现一个基于DICOM动态影像和静态影像的Streaming框架,有效地提高了浏览的实时性,使浏览影像更加方便。
关键词 医学影像系统 streaming流技术 PACS
暂未订购
基于Spark Streaming的高校网站敏感信息监测设计与实现 被引量:3
18
作者 王丹 邓谦 刘姣 《科教导刊》 2021年第5期40-41,44,共3页
高校在实现智慧化的同时也面临着网站发布信息或网页内容被黑客篡改成不符合国家或学校规定的信息及内容.通过对已有学术研究发现,现有技术的研究普遍存在着效率低、实时性差的问题,本文提出了一种基于Spark Streaming的高校网站敏感信... 高校在实现智慧化的同时也面临着网站发布信息或网页内容被黑客篡改成不符合国家或学校规定的信息及内容.通过对已有学术研究发现,现有技术的研究普遍存在着效率低、实时性差的问题,本文提出了一种基于Spark Streaming的高校网站敏感信息监测系统.该系统利用Kafka作为中间存储,系统架构在Spark Streaming框架上可实时消费Kafka中数据进行链接解析处理,将获取到的网页内容存储到Elastic Search中进行倒排索引敏感信息匹配,从而达到数据采集和数据处理同步,提高了网站监测效率. 展开更多
关键词 SPARK streaming Kafka ELASTIC Search 敏感信息监测 倒排索引
在线阅读 下载PDF
Super point detection based on sampling and data streaming algorithms
19
作者 程光 强士卿 《Journal of Southeast University(English Edition)》 EI CAS 2009年第2期224-227,共4页
In order to improve the precision of super point detection and control measurement resource consumption, this paper proposes a super point detection method based on sampling and data streaming algorithms (SDSD), and... In order to improve the precision of super point detection and control measurement resource consumption, this paper proposes a super point detection method based on sampling and data streaming algorithms (SDSD), and proves that only sources or destinations with a lot of flows can be sampled probabilistically using the SDSD algorithm. The SDSD algorithm uses both the IP table and the flow bloom filter (BF) data structures to maintain the IP and flow information. The IP table is used to judge whether an IP address has been recorded. If the IP exists, then all its subsequent flows will be recorded into the flow BF; otherwise, the IP flow is sampled. This paper also analyzes the accuracy and memory requirements of the SDSD algorithm , and tests them using the CERNET trace. The theoretical analysis and experimental tests demonstrate that the most relative errors of the super points estimated by the SDSD algorithm are less than 5%, whereas the results of other algorithms are about 10%. Because of the BF structure, the SDSD algorithm is also better than previous algorithms in terms of memory consumption. 展开更多
关键词 super point flow sampling data streaming
在线阅读 下载PDF
Spark Streaming中参数与资源协同调整策略 被引量:2
20
作者 梁毅 刘飞 +1 位作者 常仕禄 程石帆 《软件导刊》 2019年第1期45-47,55,共4页
Spark Streaming是一种典型的批量流式计算平台,可用于处理持续到达的数据流。流式数据最重要的两个特征是波动性和时效性。利用动态调整系统参数和动态调整资源满足不同数据到达速率的响应延迟,但调整参数的方式具有局限性,其用户成本... Spark Streaming是一种典型的批量流式计算平台,可用于处理持续到达的数据流。流式数据最重要的两个特征是波动性和时效性。利用动态调整系统参数和动态调整资源满足不同数据到达速率的响应延迟,但调整参数的方式具有局限性,其用户成本较大。因此提出一种参数和资源协同调整策略,采用动态邻域粒子群算法找到一种满足SLO目标且使用资源最少的系统方案。实验表明,AdaStreaming与DyBBS相比,延迟性降低了70.1%,在资源使用量上比DRA降低了42.1%。 展开更多
关键词 SPARK streaming 动态邻域粒子群 参数配置 资源分配
在线阅读 下载PDF
上一页 1 2 72 下一页 到第
使用帮助 返回顶部