Full Text

Abstract

The widespread popularity of Cloud computing as a preferred platform for the deployment of web applications have resulted in an enormous number of applications moving to the cloud, and the huge success of cloud service providers. The data center storage management plays a vital role in cloud computing environments. Especially the PC cluster-based data storage is necessary to manage data on low cost storage servers in which storage space can be reduced. The paper presents the “Map Reduce” and “Hadoop” as Big Data systems that support the processing of large sets of data in a cloud computing environment. This system presents an efficient data storage approach to push work out to many nodes in a cluster using Hadoop File System (HDFS) with variable chunk size to facilitate massive data processing and introduces the implementation enhancement on MapReduce model with BW Transform to reduce the amount of data redundancy and improves the scalability to keep on working with the amount of existing physical storage capacity when the number of users and files are increased.

References

1. Neves, Pedro Caldeira, Bradley Schmerl, Jorge Bernardino, and Javier Cámara. \"Big Data in Cloud Computing: features and issues.\"

2. Lopez, Xavier. \"Big data and advanced spatial analytics.\" In Proceedings of the 3rd International Conference on Computing for Geospatial Research and Applications, p. 5. ACM, 2012.

3. Kshetri, Nir. \"Cloud computing in developing economies.\" Computer 43, no. 10 (2010): 47-55.

4. https://en.wikipedia.org/wiki/Cloud_computing

5. Klous, Sander, and Nart Wielaard. We are Big Data: The Future of the Information Society. Springer, 2016.

6. Chandrashekar, R., Kala, M., & Mane, D. (2015). Integration of Big Data in Cloud computing environments for enhanced data processing capabilities. International Journal of Engineering Research and General Science, 240-245.

7. James Kobielus, I., & Bob Marcus, E. S. (2014). Deploying Big Data Analytics Applications to the Cloud: Roadmap for Success. Cloud Standards Customer Council.

8. A.Abouzeid1,K.B.Pawlikowski,D.Abadi,A.Silbersc hatz, A.Rasin:“ HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads”, VLDB ‘09, August 24-28, 2009, Lyon, France Copyright 2009 VLDB Endowment, ACM .

9. A.Verma, N.Zea, B.Cho, I.Gupta, and R.H.Campbell: “ Breaking the MapReduce Stage Barrier”, University of Illinois at Urbana Champaign, 2009.

10. D.Zinn, Q.Hart, T.McPhillips, B.L.ascher, Y. Simmha n, M.Giakkoupis, V.K.Prasanna: “Towards Reliable, Performant Workflows for Streaming-Applications on Cloud Platforms”, University of California,2010.

11. D. Nurmi, R. Wolski, C. Grzegorczyk, G. Obertelli, S. Soman, L. Youseff, and D. Zagorodnov:“ The Eucalyptus Open-source Cloud-computing System”, Computer Science Department, University of California, Santa Barbara, 2008.

12. J.A. Stuart, C.K. Chen, K.L. Ma: “Multi-GPU Volume Rendering using MapReduce”, MAPREDUCE 2010 Chicago, Illinois USA Copyright 2010 ACM.

13. S. Scully and W.Benjamin: “Improving Storage Efficiencies with Data Deduplication and Compression” May 2010.

14. T. Aarnio: “Parallel data processing with MapReduce”, Helsinki University of Technology, TKK T-110.5190 Seminar on Internetworking, 2009.

Keywords

Big Data, Cloud Computing, Hadoop, Map Reduce, BW Transform.