Application of Hadoop in Data Science

Abstract

Continued growth and integration of information technology in every aspect of human life has given benefits, opportunities, and challenges. The benefits lie in improved living standards and advancements, while the major challenges are to understand the information lying deep inside the raw data. For getting insight into the raw data, the emergence of Data Science has a significant role. The extraction of data, preparation for analysis, and visualization are the core activities of the Data Science domain. It uses multi-disciplinary statistical methods and techniques to obtain meaningful information from the data. From small to large scale organizations, all are in pursuit of taking benefits from the available information to aggressively compete and thrive. With the diverse application of statistical logics, Data Science has emerged as a new standard for growth by utilizing state-of-theart technologies and tools. Many software platforms and tools are contesting each other to provide advanced support and cutting-edge solutions for Data Science to dominate the market. “More is less”, seems to be a new motto nowadays to investigate the options and availability of the platforms. Among the leading platforms, Hadoop has attained a distinction and a larger market share in the Data Science field. Hadoop provides versatility, flexibility, and simplicity while being enriched in processing data using commodity clusters. The associated economy and efficient processing of the data has made it a preferred choice for stakeholders from diverse backgrounds to adapt it. Hadoop is now a de facto and optimal choice for Data Scientists. Big Data and Data Science go hand-in-hand. The former is concerned with the challenges related to analyzing data in terms of velocity, variety, veracity, and volume, while the latter tends to provide solutions utilizing various mathematical approaches and computational research. This chapter discusses the key importance of Data Science along with the utility of the Hadoop platform for processing data.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 5 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.