Spark Cloud-Based Parallel Computing for Traffic Network Flow Predictive Control Using Non-Analytical Predictive Model

Abstract : When dealing with traffic big data under the background of Internet of Things (IoT), traffic control under the single-machine computing environment is difficult to adapt to the massive and rapid analysis and decision-making. To tackle this problem, we propose a parallel computing approach of traffic network flow control based on the mechanism of model predictive control (MPC). A non-analytical rule-based traffic flow model is developed to forecast vehicle movements in the prediction horizon according to the real-time feedback information of traffic flow, and evaluate the performances of candidate control strategies. Furthermore, to accelerate the solution process of obtaining the optimal control schemes in the prediction horizon, a two-level hierarchical parallel genetic algorithm (HPGA) based on Spark cloud computing is designed. Through the parallel computing architecture, the computationally intensive optimization tasks are decomposed into multiple parallel sub-tasks with the aid of resilient distributed datasets (RDDs), which improves the computational efficiency. The simulation results demonstrate the validity of the proposed methodology for traffic network flow predictive control with respect to unsaturated and oversaturated traffic scenarios. The Spark-based parallel optimization approach has the potential to satisfy the computing requirements of online optimization when dealing with the big data of traffic network flow control while keeps favorable control performances.

 EXISTING SYSTEM :

 ? The architecture is based on systematic analysis of the requirements of the existing traffic control systems. ? However, magnitude and heterogeneity of the Big Data are beyond the capabilities of the existing approaches in ITS. ? The evolution of the existing ITS into a data-driven system has been foreseen by other researchers . ? Thus, it is crucial to explore the similarities and differences among the existing control system and stream analytics performed on Kafka. Real-time traffic control systems are composed of two main components: observation of the situation (data collection) and implementation of the selected control strategy (data processing and information dissemination).

 DISADVANTAGE :

 ? Online Internet traffic monitoring resembles a stream analytics problem, where the input is an unbounded sequence of data. ? Network traffic analysis and monitoring can be regarded as a statistics problem for a set of packets in a period of time. ? We consider TCP performance analysis as an example of how the network monitoring problem can be solved by the stream-processing method in our system. ? They presented three network monitoring applications that can be expressed as streaming analytics problems; namely, reflection attack monitoring, application performance analysis, and port scan detection. ? We demonstrated that Internet measurement and monitoring can be treated as a stream analysis problem and can be handled via a streaming processing platform.

 PROPOSED SYSTEM :

 • A substantial part of the proposed architecture has been reified in a platform prototype which relies mainly on a Kafka, an established tool for efficient processing of Big Data streams. • We are interested in high-level ITS architecture and proof of concept for smooth operation of the proposed platform, without losing generality, we have modeled this section in SUMO , a microscopic traffic simulation. • In this work, we proposed a comprehensive and flexible architecture for real-time traffic control based on Big Data analytics. • The proposed architecture has been reified in prototype a platform employing Kafka. It has been put to action in operating a feedback control loop to open or close hard shoulder of a freeway.

 ADVANTAGE :

 ? We considered the TCP performance monitoring as a special use-case of showing how network monitoring can be performed with our proposed system. ? Traditionally, Internet traffic measurement and analysis have been executed on a high-performance central server. ? However, the intermediate data of Hadoop are stored on disk (which usually has poor I/O performance); therefore, there will be dramatic performance degradation for algorithms requiring plenty of iterations. ? We implement a parallel algorithm for monitoring TCP performance parameters, such as delay and retransmission ratio with a very short delay. ? This could be used for data traffic performance analysis, such as TCP performance.