期刊文献+
共找到253篇文章
< 1 2 13 >
每页显示 20 50 100
Design and development of real-time query platform for big data based on hadoop 被引量:1
1
作者 刘小利 Xu Pandeng +1 位作者 Liu Mingliang Zhu Guobin 《High Technology Letters》 EI CAS 2015年第2期231-238,共8页
This paper designs and develops a framework on a distributed computing platform for massive multi-source spatial data using a column-oriented database(HBase).This platform consists of four layers including ETL(extract... This paper designs and develops a framework on a distributed computing platform for massive multi-source spatial data using a column-oriented database(HBase).This platform consists of four layers including ETL(extraction transformation loading) tier,data processing tier,data storage tier and data display tier,achieving long-term store,real-time analysis and inquiry for massive data.Finally,a real dataset cluster is simulated,which are made up of 39 nodes including 2 master nodes and 37 data nodes,and performing function tests of data importing module and real-time query module,and performance tests of HDFS's I/O,the MapReduce cluster,batch-loading and real-time query of massive data.The test results indicate that this platform achieves high performance in terms of response time and linear scalability. 展开更多
关键词 big data massive data storage real-time query HADOOP distributed computing
在线阅读 下载PDF
Semantic-based query processing for relational data integration 被引量:1
2
作者 苗壮 张亚非 +2 位作者 王进鹏 陆建江 周波 《Journal of Southeast University(English Edition)》 EI CAS 2011年第1期22-25,共4页
To solve the query processing correctness problem for semantic-based relational data integration,the semantics of SAPRQL(simple protocol and RDF query language) queries is defined.In the course of query rewriting,al... To solve the query processing correctness problem for semantic-based relational data integration,the semantics of SAPRQL(simple protocol and RDF query language) queries is defined.In the course of query rewriting,all relative tables are found and decomposed into minimal connectable units.Minimal connectable units are joined according to semantic queries to produce the semantically correct query plans.Algorithms for query rewriting and transforming are presented.Computational complexity of the algorithms is discussed.Under the worst case,the query decomposing algorithm can be finished in O(n2) time and the query rewriting algorithm requires O(nm) time.And the performance of the algorithms is verified by experiments,and experimental results show that when the length of query is less than 8,the query processing algorithms can provide satisfactory performance. 展开更多
关键词 data integration relational database simple protocol and RDF query language(SPARQL) minimal connectable unit query processing
在线阅读 下载PDF
Enhancing the data processing speed of a deep-learning-based three-dimensional single molecule localization algorithm (FD-DeepLoc) with a combination of feature compression and pipeline programming
3
作者 Shuhao Guo Jiaxun Lin +1 位作者 Yingjun Zhang Zhen-Li Huang 《Journal of Innovative Optical Health Sciences》 2025年第2期150-160,共11页
Three-dimensional(3D)single molecule localization microscopy(SMLM)plays an important role in biomedical applications,but its data processing is very complicated.Deep learning is a potential tool to solve this problem.... Three-dimensional(3D)single molecule localization microscopy(SMLM)plays an important role in biomedical applications,but its data processing is very complicated.Deep learning is a potential tool to solve this problem.As the state of art 3D super-resolution localization algorithm based on deep learning,FD-DeepLoc algorithm reported recently still has a gap with the expected goal of online image processing,even though it has greatly improved the data processing throughput.In this paper,a new algorithm Lite-FD-DeepLoc is developed on the basis of FD-DeepLoc algorithm to meet the online image processing requirements of 3D SMLM.This new algorithm uses the feature compression method to reduce the parameters of the model,and combines it with pipeline programming to accelerate the inference process of the deep learning model.The simulated data processing results show that the image processing speed of Lite-FD-DeepLoc is about twice as fast as that of FD-DeepLoc with a slight decrease in localization accuracy,which can realize real-time processing of 256×256 pixels size images.The results of biological experimental data processing imply that Lite-FD-DeepLoc can successfully analyze the data based on astigmatism and saddle point engineering,and the global resolution of the reconstructed image is equivalent to or even better than FD-DeepLoc algorithm. 展开更多
关键词 real-time data processing feature compression pipeline programming
原文传递
A method for improving graph queries processing using positional inverted index (P.I.I) idea in search engines and parallelization techniques 被引量:2
4
作者 Hamed Dinari Hassan Naderi 《Journal of Central South University》 SCIE EI CAS CSCD 2016年第1期150-159,共10页
The idea of positional inverted index is exploited for indexing of graph database. The main idea is the use of hashing tables in order to prune a considerable portion of graph database that cannot contain the answer s... The idea of positional inverted index is exploited for indexing of graph database. The main idea is the use of hashing tables in order to prune a considerable portion of graph database that cannot contain the answer set. These tables are implemented using column-based techniques and are used to store graphs of database, frequent sub-graphs and the neighborhood of nodes. In order to exact checking of remaining graphs, the vertex invariant is used for isomorphism test which can be parallel implemented. The results of evaluation indicate that proposed method outperforms existing methods. 展开更多
关键词 graph query processing frequent subgraph graph mining data mining positional inverted index
在线阅读 下载PDF
A real-time AI-assisted seismic monitoring system based on new nodal stations with 4G telemetry and its application in the Yangbi M_(S) 6.4 aftershock monitoring in southwest China 被引量:2
5
作者 Junlun Li Huajian Yao +10 位作者 Baoshan Wang Yang Yang Xin Hu Lishu Zhang Beng Ye Jun Yang Xiaobin Li Feng Liu Guoyi Chen Chang Guo Wen Yang 《Earthquake Research Advances》 CSCD 2022年第2期3-10,共8页
A rapidly deployable dense seismic monitoring system which is capable of transmitting acquired data in real time and analyzing data automatically is crucial in seismic hazard mitigation after a major earthquake.Howeve... A rapidly deployable dense seismic monitoring system which is capable of transmitting acquired data in real time and analyzing data automatically is crucial in seismic hazard mitigation after a major earthquake.However,it is rather difficult for current seismic nodal stations to transmit data in real time for an extended period of time,and it usually takes a great amount of time to process the acquired data manually.To monitor earthquakes in real time flexibly,we develop a mobile integrated seismic monitoring system consisting of newly developed nodal units with 4G telemetry and a real-time AI-assisted automatic data processing workflow.The integrated system is convenient for deployment and has been successfully applied in monitoring the aftershocks of the Yangbi M_(S) 6.4 earthquake occurred on May 21,2021 in Yangbi County,Dali,Yunnan in southwest China.The acquired seismic data are transmitted almost in real time through the 4G cellular network,and then processed automat-ically for event detection,positioning,magnitude calculation and source mechanism inversion.From tens of seconds to a couple of minutes at most,the final seismic attributes can be presented remotely to the end users through the integrated system.From May 27 to June 17,the real-time system has detected and located 7905 aftershocks in the Yangbi area before the internal batteries exhausted,far more than the catalog provided by China Earthquake Networks Center using the regional permanent stations.The initial application of this inte-grated real-time monitoring system is promising,and we anticipate the advent of a new era for Real-time Intelligent Array Seismology(RIAS),for better monitoring and understanding the subsurface dynamic pro-cesses caused by Earth's internal forces as well as anthropogenic activities. 展开更多
关键词 Seismic dense array 4G data transmission real-time earthquake monitoring Machine-learning assisted processing real-time intelligent array seismology
在线阅读 下载PDF
A Processing Approach for Event-Based Location Aware Queries in Hybrid Wireless Sensor Networks
6
作者 HONG Liang,LU Yansheng College of Computer Science and Technology,Huazhong University of Science and Technology,Wuhan 430074,Hubei,China 《Wuhan University Journal of Natural Sciences》 CAS 2009年第4期327-332,共6页
In hybrid wireless sensor networks, sensor mobility causes the query areas to change dynamically. Aiming at the problem of inefficiency in processing the data aggregation queries in dynamic query areas, this paper pro... In hybrid wireless sensor networks, sensor mobility causes the query areas to change dynamically. Aiming at the problem of inefficiency in processing the data aggregation queries in dynamic query areas, this paper proposes a processing approach for event-based location aware queries (ELAQ), which includes query dissemination algorithm, maximum distance projection proxy selection algorithm, in-network query propagation, and aggregation algorithm. ELAQs are triggered by the events and the query results are dependent on mobile sensors' location, which are the characteristics of ELAQ model. The results show that compared with the TinyDB query processing approach, ELAQ processing approach increases the accuracy of the query result and decreases the query response time. 展开更多
关键词 query processing wireless sensor network MOBILITY data aggregation EVENT
原文传递
A Shallow Parsing Approach to Natural Language Queries of a Database
7
作者 Richard Skeggs Stasha Lauria 《Journal of Software Engineering and Applications》 2019年第9期365-382,共18页
The performance and reliability of converting natural language into structured query language can be problematic in handling nuances that are prevalent in natural language. Relational databases are not designed to und... The performance and reliability of converting natural language into structured query language can be problematic in handling nuances that are prevalent in natural language. Relational databases are not designed to understand language nuance, therefore the question why we must handle nuance has to be asked. This paper is looking at an alternative solution for the conversion of a Natural Language Query into a Structured Query Language (SQL) capable of being used to search a relational database. The process uses the natural language concept, Part of Speech to identify words that can be used to identify database tables and table columns. The use of Open NLP based grammar files, as well as additional configuration files, assist in the translation from natural language to query language. Having identified which tables and which columns contain the pertinent data the next step is to create the SQL statement. 展开更多
关键词 NLIDB NATURAL LANGUAGE processing dataBASE query data MINING
在线阅读 下载PDF
Efficient Pr-Skyline Query Processing and Optimization in Wireless Sensor Networks
8
作者 Jianzhong Li Shuguang Xiong 《Wireless Sensor Network》 2010年第11期838-849,共12页
As one of the commonly used queries in modern databases, skyline query has received extensive attention from database research community. The uncertainty of the data in wireless sensor networks makes the corresponding... As one of the commonly used queries in modern databases, skyline query has received extensive attention from database research community. The uncertainty of the data in wireless sensor networks makes the corresponding skyline uncertain and not unique. This paper investigates the Pr-Skyline problem, i.e., how to compute the skyline with the highest existence probability in a computational and energy-efficient way. We formulate the problem and prove that it is NP-Complete and cannot be approximated in a given expression. However, the proposed algorithm SKY-SEARCH with pruning techniques can guarantee the computational efficiency given relatively large input size, while the filter-based distributed optimization strategy significantly reduces the transmission cost and the required storage space of the sensor nodes. Extensive experiments verify the efficiency and scalability of SKY-SEARCH and the distributed optimizing strategy. 展开更多
关键词 Wireless Sensor Network query processing UNCERTAIN data PROBABILISTIC data SKYLINE query
在线阅读 下载PDF
Optimizing Query Results Integration Process Using an Extended Fuzzy C-Means Algorithm
9
作者 Naoual Mouhni Abderrafiaa Elkalay Mohamed Chakraoui 《Journal of Software Engineering and Applications》 2014年第5期354-359,共6页
Cleaning duplicate data is a major problem that persists even though many works have been done to solve it, due to the exponential growth of data amount treated and the necessity to use scalable and speed algorithms. ... Cleaning duplicate data is a major problem that persists even though many works have been done to solve it, due to the exponential growth of data amount treated and the necessity to use scalable and speed algorithms. This problem depends on the type and quality of data, and differs according to the volume of data set manipulated. In this paper we are going to introduce a novel framework based on extended fuzzy C-means algorithm by using topic ontology. This work aims to improve the OLAP querying process over heterogeneous data warehouses that contain big data sets, by improving query results integration, eliminating redundancies by using the extended classification algorithm, and measuring the loss of information. 展开更多
关键词 Clustering Classification and Association RULES dataBASE Integration data WAREHOUSE and REPOSITORY Heterogeneous dataBASES query processing
在线阅读 下载PDF
Patient Centered Real-Time Mobile Health Monitoring System
10
作者 Won-Jae Yi Jafar Saniie 《E-Health Telecommunication Systems and Networks》 2016年第4期75-94,共20页
In this paper, we introduce a system architecture for a patient centered mobile health monitoring (PCMHM) system that deploys different sensors to determine patients’ activities, medical conditions, and the cause of ... In this paper, we introduce a system architecture for a patient centered mobile health monitoring (PCMHM) system that deploys different sensors to determine patients’ activities, medical conditions, and the cause of an emergency event. This system combines and analyzes sensor data to produce the patients’ detailed health information in real-time. A central computational node with data analyzing capability is used for sensor data integration and analysis. In addition to medical sensors, surrounding environmental sensors are also utilized to enhance the interpretation of the data and to improve medical diagnosis. The PCMHM system has the ability to provide on-demand health information of patients via the Internet, track real-time daily activities and patients’ health condition. This system also includes the capability for assessing patients’ posture and fall detection. 展开更多
关键词 Patient Remote Health Monitoring real-time Sensor data processing Wireless Body Sensor Network Fall Detection Heart Monitoring
在线阅读 下载PDF
Research on the Development Strategies of Realtime Data Analysis and Decision-support Systems
11
作者 Wei Tang 《Journal of Electronic Research and Application》 2025年第2期204-210,共7页
With the advent of the big data era,real-time data analysis and decision-support systems have been recognized as essential tools for enhancing enterprise competitiveness and optimizing the decision-making process.This... With the advent of the big data era,real-time data analysis and decision-support systems have been recognized as essential tools for enhancing enterprise competitiveness and optimizing the decision-making process.This study aims to explore the development strategies of real-time data analysis and decision-support systems,and analyze their application status and future development trends in various industries.The article first reviews the basic concepts and importance of real-time data analysis and decision-support systems,and then discusses in detail the key technical aspects such as system architecture,data collection and processing,analysis methods,and visualization techniques. 展开更多
关键词 real-time data analysis Decision-support system Big data System architecture data processing Visualization technology
在线阅读 下载PDF
Explicit ARL Computational for a Modified EWMA Control Chart in Autocorrelated Statistical Process Control Models
12
作者 Yadpirun Supharakonsakun Yupaporn Areepong Korakoch Silpakob 《Computer Modeling in Engineering & Sciences》 2025年第10期699-720,共22页
This study presents an innovative development of the exponentially weighted moving average(EWMA)control chart,explicitly adapted for the examination of time series data distinguished by seasonal autoregressive moving ... This study presents an innovative development of the exponentially weighted moving average(EWMA)control chart,explicitly adapted for the examination of time series data distinguished by seasonal autoregressive moving average behavior—SARMA(1,1)L under exponential white noise.Unlike previous works that rely on simplified models such as AR(1)or assume independence,this research derives for the first time an exact two-sided Average Run Length(ARL)formula for theModified EWMAchart under SARMA(1,1)L conditions,using a mathematically rigorous Fredholm integral approach.The derived formulas are validated against numerical integral equation(NIE)solutions,showing strong agreement and significantly reduced computational burden.Additionally,a performance comparison index(PCI)is introduced to assess the chart’s detection capability.Results demonstrate that the proposed method exhibits superior sensitivity to mean shifts in autocorrelated environments,outperforming existing approaches.The findings offer a new,efficient framework for real-time quality control in complex seasonal processes,with potential applications in environmental monitoring and intelligent manufacturing systems. 展开更多
关键词 Statistical process control average run length modified EWMA control chart autocorrelated data SARMA process computational modeling real-time monitoring
在线阅读 下载PDF
Data partitioning based on sampling for power load streams
13
作者 王永利 徐宏炳 +2 位作者 董逸生 钱江波 刘学军 《Journal of Southeast University(English Edition)》 EI CAS 2005年第3期293-298,共6页
A novel data streams partitioning method is proposed to resolve problems of range-aggregation continuous queries over parallel streams for power industry.The first step of this method is to parallel sample the data,wh... A novel data streams partitioning method is proposed to resolve problems of range-aggregation continuous queries over parallel streams for power industry.The first step of this method is to parallel sample the data,which is implemented as an extended reservoir-sampling algorithm.A skip factor based on the change ratio of data-values is introduced to describe the distribution characteristics of data-values adaptively.The second step of this method is to partition the fluxes of data streams averagely,which is implemented with two alternative equal-depth histogram generating algorithms that fit the different cases:one for incremental maintenance based on heuristics and the other for periodical updates to generate an approximate partition vector.The experimental results on actual data prove that the method is efficient,practical and suitable for time-varying data streams processing. 展开更多
关键词 data streams continuous queries parallel processing sampling data partitioning
在线阅读 下载PDF
Fast Web - Based Data Transmission 被引量:2
14
作者 Wei Zukuan Department of Computer Science & Engineering, Inha University, Inchon 402 751, Korea Kim Jaehong Department of Computer Science, Youngdong University, Youngdong, Korea Bae Haeyoung Department of Computer Science & Engineering, Inha Uni 《Journal of China University of Geosciences》 SCIE CSCD 2001年第2期165-176,共12页
Since web based GIS processes large size spatial geographic information on internet, we should try to improve the efficiency of spatial data query processing and transmission. This paper presents two efficient metho... Since web based GIS processes large size spatial geographic information on internet, we should try to improve the efficiency of spatial data query processing and transmission. This paper presents two efficient methods for this purpose: division transmission and progressive transmission methods. In division transmission method, a map can be divided into several parts, called “tiles”, and only tiles can be transmitted at the request of a client. In progressive transmission method, a map can be split into several phase views based on the significance of vertices, and a server produces a target object and then transmits it progressively when this spatial object is requested from a client. In order to achieve these methods, the algorithms, “tile division”, “priority order estimation” and the strategies for data transmission are proposed in this paper, respectively. Compared with such traditional methods as “map total transmission” and “layer transmission”, the web based GIS data transmission, proposed in this paper, is advantageous in the increase of the data transmission efficiency by a great margin. 展开更多
关键词 spatial data transmission spatial query processing web based GIS geographic information system spatial database.
在线阅读 下载PDF
Hash-area-based data dissemination protocol in wireless sensor networks 被引量:1
15
作者 王田 王国军 +1 位作者 过敏意 贾维嘉 《Journal of Central South University of Technology》 EI 2008年第3期392-398,共7页
HashQuery,a Hash-area-based data dissemination protocol,was designed in wireless sensor networks. Using a Hash function which uses time as the key,both mobile sinks and sensors can determine the same Hash area. The se... HashQuery,a Hash-area-based data dissemination protocol,was designed in wireless sensor networks. Using a Hash function which uses time as the key,both mobile sinks and sensors can determine the same Hash area. The sensors can send the information about the events that they monitor to the Hash area and the mobile sinks need only to query that area instead of flooding among the whole network,and thus much energy can be saved. In addition,the location of the Hash area changes over time so as to balance the energy consumption in the whole network. Theoretical analysis shows that the proposed protocol can be energy-efficient and simulation studies further show that when there are 5 sources and 5 sinks in the network,it can save at least 50% energy compared with the existing two-tier data dissemination(TTDD) protocol,especially in large-scale wireless sensor networks. 展开更多
关键词 wireless sensor networks Hash function data dissemination query processing mobile sinks
在线阅读 下载PDF
Research on Welding Quality Traceability Model of Offshore Platform Block Construction Process
16
作者 Jinghua Li Wenhao Yin +1 位作者 Boxin Yang Qinghua Zhou 《Computer Modeling in Engineering & Sciences》 SCIE EI 2023年第1期699-730,共32页
Quality traceability plays an essential role in assembling and welding offshore platform blocks.The improvement of the welding quality traceability system is conducive to improving the durability of the offshore platf... Quality traceability plays an essential role in assembling and welding offshore platform blocks.The improvement of the welding quality traceability system is conducive to improving the durability of the offshore platform and the process level of the offshore industry.Currently,qualitymanagement remains in the era of primary information,and there is a lack of effective tracking and recording of welding quality data.When welding defects are encountered,it is difficult to rapidly and accurately determine the root cause of the problem from various complexities and scattered quality data.In this paper,a composite welding quality traceability model for offshore platform block construction process is proposed,it contains the quality early-warning method based on long short-term memory and quality data backtracking query optimization algorithm.By fulfilling the training of the early-warning model and the implementation of the query optimization algorithm,the quality traceability model has the ability to assist enterprises in realizing the rapid identification and positioning of quality problems.Furthermore,the model and the quality traceability algorithm are checked by cases in actual working conditions.Verification analyses suggest that the proposed early-warningmodel for welding quality and the algorithmfor optimizing backtracking requests are effective and can be applied to the actual construction process. 展开更多
关键词 Quality traceability model block construction process welding quality management long short-term memory quality data backtracking query optimization algorithm
在线阅读 下载PDF
Automated Performance Tuning of Data Management Systems with Materializations and Indices
17
作者 Nan N. Noon Janusz R. Getta 《Journal of Computer and Communications》 2016年第5期46-52,共7页
Automated performance tuning of data management systems offer various benefits such as improved performance, declined administration costs, and reduced workloads to database administrators (DBAs). Currently, DBAs tune... Automated performance tuning of data management systems offer various benefits such as improved performance, declined administration costs, and reduced workloads to database administrators (DBAs). Currently, DBAs tune the performance of database systems with a little help from the database servers. In this paper, we propose a new technique for automated performance tuning of data management systems. Firstly, we show how to use the periods of low workload time for performance improvements in the periods of high workload time. We demonstrate that extensions of a database system with materialised views and indices when a workload is low may contribute to better performance for a successive period of high workload. The paper proposes several online algorithms for continuous processing of estimated database workloads and for the discovery of the best plan for materialised view and index database extensions and of elimination of the extensions that are no longer needed. We present the results of experiments that show how the proposed automated performance tuning technique improves the overall performance of a data management system.   展开更多
关键词 Automated Performance Tuning query processing MATERIALIZATION INDEXING data Management Systems
在线阅读 下载PDF
时空数据查询技术研究综述 被引量:1
18
作者 孟祥福 翁雪 徐永杰 《计算机科学与探索》 北大核心 2025年第8期2001-2023,共23页
随着现代信息技术的快速发展与应用,时空数据的规模迅速增长。这些数据呈现出海量聚集、高维异构以及动态复杂等特点。近年来,以时空数据为背景的时空查询技术得到广泛的研究和应用,如何有效地存储、管理和查询这些数据成为了研究的重... 随着现代信息技术的快速发展与应用,时空数据的规模迅速增长。这些数据呈现出海量聚集、高维异构以及动态复杂等特点。近年来,以时空数据为背景的时空查询技术得到广泛的研究和应用,如何有效地存储、管理和查询这些数据成为了研究的重点。对时空数据的相关查询技术进行综述,从时空数据相关基本概念入手,系统阐述了当前主流的时空查询处理模式,涵盖了范围查询、K近邻查询、反K近邻查询等多种类型;介绍了不同的时空索引技术,包括基于轨迹的索引结构、基于抽样的索引以及其他创新的索引方法;分析了结合其他技术的查询方法,主要包括时空-文本查询、语义近似轨迹查询、并行和分布式查询等,这些技术不仅提升了时空查询的多样性和准确性,还能有效地处理大规模时空数据。展望了时空查询技术的未来发展方向,包括查询结果的可视化展示、隐私保护以及结合机器学习的新型索引结构,为时空数据的高效利用提供了新的思路和挑战。 展开更多
关键词 时空数据 查询处理 索引技术 时空-文本 语义近似 分布式
在线阅读 下载PDF
机器学习赋能的多维数据查询处理研究综述 被引量:4
19
作者 马超红 郝新丽 +1 位作者 孟小峰 张旭康 《计算机学报》 北大核心 2025年第1期100-123,共24页
多维数据的查询和处理在数据库中普遍存在。高效的多维数据查询处理,一方面依赖于精细的索引结构,例如R-tree、KD-tree等被广泛应用;另一方面,也有诸多工作探索利用硬件优势设计高效的数据布局,即研究面向扫描的数据处理策略以及构建数... 多维数据的查询和处理在数据库中普遍存在。高效的多维数据查询处理,一方面依赖于精细的索引结构,例如R-tree、KD-tree等被广泛应用;另一方面,也有诸多工作探索利用硬件优势设计高效的数据布局,即研究面向扫描的数据处理策略以及构建数据概要,避免高代价地访问原始数据。然而,随着数字化社会的发展,网络Web服务更加普及,传感器网络无处不在,诸如网约车、电子地图等基于位置的服务愈发盛行,使得多维数据正在以前所未有的速度产生,对查询处理提出新的要求,包括更快的查询响应、更低的存储占用。近年来,机器学习包括深度学习算法不断优化,且计算机等硬件环境持续发展,为多维数据查询处理带来更多的优化契机,不仅降低查询执行时间,同时能够节省存储资源,取得显著性优势。因此,机器学习被广泛应用于构建更好的数据管理和数据分析任务解决方案。该文提出机器学习赋能的多维数据查询处理研究框架,一方面介绍机器学习模型对多维索引结构的优化和改进;另一方面,介绍机器学习对不依赖索引结构的查询处理任务的赋能研究,包括数据布局策略和数据概要研究。在总结已有研究现状的基础上,指出该领域面临的挑战和未来研究方向。 展开更多
关键词 查询处理 多维学习化索引 数据布局 数据概要 机器学习
在线阅读 下载PDF
基于StarRocks的实时物联网数据处理系统 被引量:2
20
作者 董一舟 潘伟华 +1 位作者 张楠 孟壮 《计算机与现代化》 2025年第1期15-19,共5页
随着物联网技术的普及和应用,大量的实时数据需要被处理和分析,因为物联数据的海量性、实时性特点,传统数据库无法满足其数据存储规模和数据处理效率的要求。本文提出一种基于StarRocks的分布式实时物联网数据处理系统。该系统利用StarR... 随着物联网技术的普及和应用,大量的实时数据需要被处理和分析,因为物联数据的海量性、实时性特点,传统数据库无法满足其数据存储规模和数据处理效率的要求。本文提出一种基于StarRocks的分布式实时物联网数据处理系统。该系统利用StarRocks的分布式架构构建底层数据存储,通过引入消息队列和数据合并批量提交技术,保证数据的快速写入;同时通过存储策略优化、索引优化、物化视图技术,实现对大规模实时数据的快速处理和查询;系统强大的数据压缩能力也有效节省了数据存储空间。该框架在数据存储规模上支持横向扩展,提高了可用性和健壮性。通过实验分析,该系统在数据写入、数据查询、数据压缩方面较传统分布式数据库具有明显优势。 展开更多
关键词 StarRocks 实时数据处理 分布式系统 数据压缩 查询优化
在线阅读 下载PDF
上一页 1 2 13 下一页 到第
使用帮助 返回顶部