Design of Distributed Network Mass Data Processing System based on Cloud Computing Technology

Abstract : In recent years, with the rapid development of cloud computing industry, big data processing technology is also constantly mature. At the same time, the scale of domestic mobile Internet market continues to expand, with more than 500 million users, bringing huge mobile Internet business data. In this context, the method of large-scale network data processing based on cloud computing big data processing technology is a very valuable research topic. From the perspective of the current situation of mobile Internet, on the one hand, mobile data services have risen sharply. And that puts a lot of pressure on the operator. Moreover, they need to invest more money in network construction and upgrading. Traditional voice and other SMS services are down. The operators then reached a point where they did not increase their earnings. Therefore, it is very important for communication operators to study the usage of large-scale mobile Internet data service resources obtained through service channels.

 EXISTING SYSTEM :

 ? Cloud providers must use and adopt existing standards wherever appropriate. The IT industry has invested heavily in existing standards and standards organizations; there is no need to duplicate or reinvent them. ? When new standards (or adjustments to existing standards) are needed, we must be judicious and pragmatic to avoid creating too many standards. ? Though the above said challenges are existing in cloud computing, security is the major concern. Security in terms of data handling, maintaining, migrating or virtualization in clouds. ? Cloud providers must not use their market position to lock customers into their particular platforms and limiting their choice of providers.

 DISADVANTAGE :

 ? Security is an important problem that limits the development and popularization of cloud computing technology. ? Researchers have done a lot of research on the problem, but after the introduction of large-scale network data processing project of mobile Internet, the existing Hadoop standard is still the security detection of data flow, which needs to be strengthened. ? The network security of this platform, the requirement of high speed and high precision of anomaly detection, the original network security technology with data encryption and security authentication, can not solve the security problem in this large-scale network data processing platform.

 PROPOSED SYSTEM :

 • The purpose of this paper is to explore key concepts and ideas surrounding the security challenges of cloud computing, with particular reference to configuration having considerable impact on securing cloud services. • The main purpose of our paper is to present a basic framework and determine benefits from Cloud Computing as an alternative to conventional IT infrastructure, such as privately owned and managed IT hardware. • With this paper we have tried to find out the security issues and obstacles in the cloud and appropriate solutions for obstacles. • Our purpose to this paper is to represent the Challenges and issues to be considered when switching to virtual cloud in IT industry.

 ADVANTAGE :

 ? The processing has the characteristics of high-speed transmission, huge data volume and half structure. ? HDFS technology realizes efficient and fast distributed storage of large amount of network data, and provides subsequent data processing services based on MapReduce technology. ? Based on the mobile Internet, people can access the Internet through mobile communication network in various mobile terminals such as mobile phones and tablets, and various network services such as language, video, social network and e-commerce can be used. ? In order to realize the communication and distribution of control information and original data, TCP connection is used.