The Internet of Things(IoT)has revolutionized our world today by providing greater levels of accessibility,connectivity and ease to our everyday lives.It enables massive amounts of data to be traversed across multiple...The Internet of Things(IoT)has revolutionized our world today by providing greater levels of accessibility,connectivity and ease to our everyday lives.It enables massive amounts of data to be traversed across multiple heterogeneous devices that are all interconnected.This phenomenon makes IoT networks vulnerable to various network attacks and intrusions.Building an Intrusion Detection System(IDS)for IoT networks is challenging as they enable a massive amount of data to be aggregated,which is difficult to handle and analyze in real time mainly because of the heterogeneous nature of IoT devices.This inefficient,traditional IDS approach accentuates the need to develop advanced IDS techniques by employing Machine or Deep Learning.This paper presents a deep ensemble-based IDS using Lambda architecture by following a multi-pronged classification approach.Binary classification uses Long Short Term Memory(LSTM)to differentiate between malicious and benign traffic,while the multi-class classifier uses an ensemble of LSTM,Convolutional Neural Network and Artificial Neural Network classifiers to detect the type of attacks.The model training is performed in the batch layer,while real-time evaluation is carried out through model inferences in the speed layer of the Lambda architecture.The proposed approach gives high accuracy of over 99.93%and saves useful processing time due to the multi-pronged classification strategy and using the lambda architecture.展开更多
This paper focuses on facilitating state-of-the-art applications of big data analytics(BDA) architectures and infrastructures to telecommunications(telecom) industrial sector.Telecom companies are dealing with terabyt...This paper focuses on facilitating state-of-the-art applications of big data analytics(BDA) architectures and infrastructures to telecommunications(telecom) industrial sector.Telecom companies are dealing with terabytes to petabytes of data on a daily basis. Io T applications in telecom are further contributing to this data deluge. Recent advances in BDA have exposed new opportunities to get actionable insights from telecom big data. These benefits and the fast-changing BDA technology landscape make it important to investigate existing BDA applications to telecom sector. For this, we initially determine published research on BDA applications to telecom through a systematic literature review through which we filter 38 articles and categorize them in frameworks, use cases, literature reviews, white papers and experimental validations. We also discuss the benefits and challenges mentioned in these articles. We find that experiments are all proof of concepts(POC) on a severely limited BDA technology stack(as compared to the available technology stack), i.e.,we did not find any work focusing on full-fledged BDA implementation in an operational telecom environment. To facilitate these applications at research-level, we propose a state-of-the-art lambda architecture for BDA pipeline implementation(called Lambda Tel) based completely on open source BDA technologies and the standard Python language, along with relevant guidelines.We discovered only one research paper which presented a relatively-limited lambda architecture using the proprietary AWS cloud infrastructure. We believe Lambda Tel presents a clear roadmap for telecom industry practitioners to implement and enhance BDA applications in their enterprises.展开更多
As an important service model for advanced computing,SaaS uses a defined protocol that manages services and applications.The popularity of advanced computing has reached a level that has led to the generation of large...As an important service model for advanced computing,SaaS uses a defined protocol that manages services and applications.The popularity of advanced computing has reached a level that has led to the generation of large data sets,which is also called Big data.Big data is evolving with great velocity,large volumes,and great diversity.Such an amplification of data has brought into question the existing database tools in terms of their capabilities.Previously,storage and processing of data were simple tasks;however,it is now one of the biggest challenges in the industry.Experts are paying close attention to big data.Designing a system capable of storing and analyzing such data in order to extract meaningful information for decision-making is a priority.The Apache Hadoop,Spark,and NoSQL databases are some of the core technologies that are being used to solve these issues.This paper contributes to the solutions to the issues of big data storage and processing.It presents an analysis of the current technologies in the industry that could be useful in this context.Efforts have been focused on implementing a novel Trinity model,which is built using the lambda architecture with the following technologies:Hadoop,Spark,Kafka,and MongoDB.展开更多
文摘The Internet of Things(IoT)has revolutionized our world today by providing greater levels of accessibility,connectivity and ease to our everyday lives.It enables massive amounts of data to be traversed across multiple heterogeneous devices that are all interconnected.This phenomenon makes IoT networks vulnerable to various network attacks and intrusions.Building an Intrusion Detection System(IDS)for IoT networks is challenging as they enable a massive amount of data to be aggregated,which is difficult to handle and analyze in real time mainly because of the heterogeneous nature of IoT devices.This inefficient,traditional IDS approach accentuates the need to develop advanced IDS techniques by employing Machine or Deep Learning.This paper presents a deep ensemble-based IDS using Lambda architecture by following a multi-pronged classification approach.Binary classification uses Long Short Term Memory(LSTM)to differentiate between malicious and benign traffic,while the multi-class classifier uses an ensemble of LSTM,Convolutional Neural Network and Artificial Neural Network classifiers to detect the type of attacks.The model training is performed in the batch layer,while real-time evaluation is carried out through model inferences in the speed layer of the Lambda architecture.The proposed approach gives high accuracy of over 99.93%and saves useful processing time due to the multi-pronged classification strategy and using the lambda architecture.
基金supported in part by the Big Data Analytics Laboratory(BDALAB)at the Institute of Business Administration under the research grant approved by the Higher Education Commission of Pakistan(www.hec.gov.pk)the Darbi company(www.darbi.io)
文摘This paper focuses on facilitating state-of-the-art applications of big data analytics(BDA) architectures and infrastructures to telecommunications(telecom) industrial sector.Telecom companies are dealing with terabytes to petabytes of data on a daily basis. Io T applications in telecom are further contributing to this data deluge. Recent advances in BDA have exposed new opportunities to get actionable insights from telecom big data. These benefits and the fast-changing BDA technology landscape make it important to investigate existing BDA applications to telecom sector. For this, we initially determine published research on BDA applications to telecom through a systematic literature review through which we filter 38 articles and categorize them in frameworks, use cases, literature reviews, white papers and experimental validations. We also discuss the benefits and challenges mentioned in these articles. We find that experiments are all proof of concepts(POC) on a severely limited BDA technology stack(as compared to the available technology stack), i.e.,we did not find any work focusing on full-fledged BDA implementation in an operational telecom environment. To facilitate these applications at research-level, we propose a state-of-the-art lambda architecture for BDA pipeline implementation(called Lambda Tel) based completely on open source BDA technologies and the standard Python language, along with relevant guidelines.We discovered only one research paper which presented a relatively-limited lambda architecture using the proprietary AWS cloud infrastructure. We believe Lambda Tel presents a clear roadmap for telecom industry practitioners to implement and enhance BDA applications in their enterprises.
文摘As an important service model for advanced computing,SaaS uses a defined protocol that manages services and applications.The popularity of advanced computing has reached a level that has led to the generation of large data sets,which is also called Big data.Big data is evolving with great velocity,large volumes,and great diversity.Such an amplification of data has brought into question the existing database tools in terms of their capabilities.Previously,storage and processing of data were simple tasks;however,it is now one of the biggest challenges in the industry.Experts are paying close attention to big data.Designing a system capable of storing and analyzing such data in order to extract meaningful information for decision-making is a priority.The Apache Hadoop,Spark,and NoSQL databases are some of the core technologies that are being used to solve these issues.This paper contributes to the solutions to the issues of big data storage and processing.It presents an analysis of the current technologies in the industry that could be useful in this context.Efforts have been focused on implementing a novel Trinity model,which is built using the lambda architecture with the following technologies:Hadoop,Spark,Kafka,and MongoDB.
