Analysis of How Scalable Features in Hadoop / MapReduce by Internet Traffic Management

Deden Witarsyah Jacob, Muhammed.E Abd Alkhalec Tharwat, Mohd Farhan Md Fudzee, Azizul Azhar Ramli, Shahreen Kasim, Muharman Lubis

Abstract


Internet traffic monitoring is to measure and analyze the network bottlenecks to manage the online data are transferring processes efficiently. Various tools have been developed by using internet traffic measurement and internet traffic analysis tools, such as Hadoop. Activity measurement and adaptive examination represent the dynamics of information exchange. On the other hand, information exchange and dynamics measure movement in light of the system assets that can be accessed depending on the characteristics of the exchanged information. The main aim of this work is to apply scalable features of internet traffic measurement and analysis using Hadoop to understand the effects of these features on the speed of transferring data. This gives a new vision or opportunity to dynamically adapting the most suitable traffic measurement and analysis feature according to network capabilities and environment. This research employs Hadoop/Map Reduce as scalable internet traffic measurement and analysis tools. The simulation was conducted by using five personal computers; one as a server and four virtual computers as network nodes. Each computer has 2GB memory and 100GB storage. Five types of data segmentation are utilized 10 MB, 40MB, 64MB, 200MB, and500MB. The speed of the network is calculating in a megabit per second (Mbs) based upon the network speed on the number of allocated PCs (100 Mbs/4). The simulation is conducted to test the data transfer time based on various selections of network capabilities such as transferring extensive data through a network of medium and heavy usage.

Keywords


Internet traffic measurement; traffic network; data transfer and sharing; Hadoop; Map Reduce.

Full Text:

PDF

References


J. Ni, W. Weng, J. Chen, and K. Lei, "Internet Traffic Analysis Using Community Detection and Apache Spark," in Proceedings - 2017 Int. Conf.e on Cyber-Enabled Distributed Computing and Knowledge Discovery, CyberC 2017, 2017, vol. 2018-Janua, pp. 213–219, doi: 10.1109/CyberC.2017.85.

E. Nazari, M. H. Shahriari, and H. Tabesh, "Big Data Analysis in Healthcare: Apache Hadoop , Apache spark and Apache Flink," Front. Heal. Informatics, vol. 8, no. 1, p. 14, 2019, doi: 10.30699/fhi.v8i1.180.

R. Y. Zhong, S. T. Newman, G. Q. Huang, and S. Lan, "Big Data for supply chain management in the service and manufacturing sectors: Challenges, opportunities, and future perspectives," Comput. Ind. Eng., vol. 101, pp. 572–591, 2016, doi: 10.1016/j.cie.2016.07.013.

L. T. Ibrahim, R. Hassan, K. Ahmad, A. N. Asat, and H. Omar, "Online traffic measurement and analysis in big data: Comparative research review," Am. J. Appl. Sci., vol. 13, no. 4, pp. 420–431, 2016, doi: 10.3844/ajassp.2016.420.431.

D. W. Jacob, M. F. M. Fudzee, M. A. Salamat, "A Conceptual Study on Generic End-Users Adoption of e-Government Services," Int. Journal on Advanced Science, Engineering and Information Technology, 7 (3), pp.1000-1006, 2017.

L. T. Ibrahim, R. Hassan, K. Ahmad, and A. N. Asat, "A study on improvement of internet traffic measurement and analysis using Hadoop system," in 2015 International Conference on Electrical Engineering and Informatics (ICEEI), 2015, pp. 462–466, doi: 10.1109/ICEEI.2015.7352545.

H. Lu, C. Hai-Shan, and H. Ting-Ting, "Research on Hadoop Cloud Computing Model and its Applications," 2012 Third Int. Conf. Netw. Distrib. Comput., pp. 59–63, 2012, doi: 10.1109/ICNDC.2012.22.

H. Cui, Y. Yao, K. Zhang, F. Sun, and Y. Liu, "Network Traffic Prediction Based on Hadoop," in 2014 International Symposium on Wireless Personal Multimedia Communications (WPMC), 2014, no. c, pp. 29–33, doi: 10.1109/WPMC.2014.7014785.

F. Dongyu, L. Zhu, and Z. Lei, "Review of hadoop performance optimization," in 2016 2nd IEEE International Conference on Computer and Communications (ICCC), 2017, pp. 65–68, doi: 10.1109/CompComm.2016.7924666.

K. Jeřábek and O. Ryšavý, "Big data network flow processing using apache spark," in ACM International Conference Proceeding Series, 2019, pp. 1–9, doi: 10.1145/3352700.3352709.

Y. Liu, H. Lu, X. Li, and D. Zhao, "An approach for service function chain reconfiguration in network function virtualization architectures," IEEE Access, vol. 7, pp. 147224–147237, 2019, doi: 10.1109/ACCESS.2019.2946648.

J. Abawajy, M. F. M. Fudzee, M. M. Hassan, M. Alrubalan, " Service Level Agreement Management Framework for Utility-oriented Computing Platform" Journal of Computing, 71 (11), pp. 4287-4303, 2015.

R. Pries, F. Wamser, D. Staehle, K. Heck, and P. Tran-Gia, "Traffic Measurement and Analysis of a Broadband Wireless Internet Access," in VTC Spring 2009 - IEEE 69th Vehicular Technology Conference, 2009, pp. 1–5, doi: 10.1109/VETECS.2009.5073890.

B. Xu and S. A. Kumar, "Big Data Analytics Framework for System Health Monitoring," in Proceedings - 2015 IEEE International Congress on Big Data, BigData Congress 2015, 2015, pp. 401–408, doi: 10.1109/BigDataCongress.2015.66.

Y. Lee & Y. Lee, "Toward scalable internet traffic measurement and analysis with Hadoop," ACM SIGCOMM Comput. Commun. Rev., vol. 43, no. 1, pp. 5–13, 2013, doi: 10.1145/2427036.2427038.

N. A. Liu & J. Feng Liu, "Monitoring and analyzing big traffic data of a large-scale cellular network with Hadoop," IEEE Netw., vol. 28, no. 4, pp. 32–39, 2014, doi: 10.1109/MNET.2014.6863129.

D. Quick and K. K. R. Choo, "Digital droplets: Microsoft SkyDrive forensic data remnants," Futur. Gener. Comput. Syst., vol. 29, no. 6, pp. 1378–1394, 2013, doi: 10.1016/j.future.2013.02.001.

J. Yang, L. Wang, A. Lesh, and B. Lockerbie, "Manipulating network traffic to evade stepping-stone intrusion detection," Internet of Things, vol. 3–4, pp. 34–45, 2018, doi: 10.1016/j.iot.2018.08.011.

L. U. Laboshin, A. A. Lukashin, and V. S. Zaborovsky, "The Big Data Approach to Collecting and Analyzing Traffic Data in Large Scale Networks," Procedia Comput. Sci., vol. 103, no. October 2016, pp. 536–542, 2017, doi: 10.1016/j.procs.2017.01.048.

A. Ibitola and O. B. Longe, "Analysis and Evaluation of Network-Based Intrusion Detection and Prevention System in an Enterprise Network Using Snort Freeware," African J. Comput. ICT, vol. 6, no. 2, pp. 169–184, 2013.

A. Finamore, M. Mellia, M. Meo, M. M. Munafo & D. Rossi, "Experiences of Internet traffic monitoring with tstat," IEEE Netw., vol. 25, no. 3, pp. 1–7, 2011.

D. Adami, C. Callegari, S. Giordano & M. Pagano, “Skypeâ€Hunter: A realâ€time system for the detection and classification of Skype traffic,†Int. J. Commun. Syst., vol. 25, no. 3, pp. 386–403, 2012.

C. Shen and L. Huang, "On detection accuracy of L7-filter and OpenDPI," in 2012 Third International Conference on Networking and Distributed Computing, 2012, pp. 119–123.

A. Bakhshandeh, "An efficient user identification approach based on Netflow analysis," in 2018 15th Int. ISC (Iranian Society of Cryptology) Conf. on Information Security and Cryptology (ISCISC), 2018, pp. 1–5.

G. Poppe, F., Van den Bosch, S., de La Vallée-Poussin, P., Van Hove, H., De Neve, H., & Petit, "Choosing the objectives for traffic engineering in IP backbone networks based on Quality-of-Service requirements," in International Workshop on Quality of Future Internet Services, 2000, pp. 129–140.




DOI: http://dx.doi.org/10.18517/ijaseit.13.1.10882

Refbacks

  • There are currently no refbacks.


Published by INSIGHT - Indonesian Society for Knowledge and Human Development