Inference patterns from Big Data using aggregation, filtering and tagging- A survey

Pathak Anand Prakashbhai, Hari Mohan Pandey

Research output: Chapter in Book/Report/Conference proceedingConference proceeding (ISBN)

3 Citations (Scopus)

Abstract

This paper reviews various approaches to infer the patterns from Big Data using aggregation, filtering and tagging. Earlier research shows that data aggregation concerns about gathered data and how efficiently it can be utilized. It is understandable that at the time of data gathering one does not care much about whether the gathered data will be useful or not. Hence, filtering and tagging of the data are the crucial steps in collecting the relevant data to fulfill the need. Therefore the main goal of this paper is to present a detailed and comprehensive survey on different approaches. To make the concept clearer, we have provided a brief introduction of Big Data, how it works, working of two data aggregation tools (namely, flume and sqoop), data processing tools (hive and mahout) and various algorithms that can be useful to understand the topic. At last we have included comparisons between aggregation tools, processing tools as well as various algorithms through its pre-process, matching time, results and reviews.

Original languageEnglish
Title of host publicationProceedings of the 5th International Conference on Confluence 2014: The Next Generation Information Technology Summit
Subtitle of host publicationThe Next Generation Information Technology Summit
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages66-71
Number of pages6
ISBN (Electronic)9781479942367
ISBN (Print)9781479942367
DOIs
Publication statusPublished - 6 Nov 2014
Event5th International Conference on Confluence 2014 - The Next Generation Information Technology Summit - Noida, India
Duration: 25 Sep 201426 Sep 2014

Publication series

NameProceedings of the 5th International Conference on Confluence 2014: The Next Generation Information Technology Summit

Conference

Conference5th International Conference on Confluence 2014 - The Next Generation Information Technology Summit
CountryIndia
CityNoida
Period25/09/1426/09/14

Fingerprint

Agglomeration
Big data
Processing

Keywords

  • aggregation
  • Big Data
  • filtering
  • Flume
  • Hadoop
  • HDFS
  • Hive
  • Mahout
  • MapReduce
  • Sqoop
  • tagging

Cite this

Prakashbhai, P. A., & Pandey, H. M. (2014). Inference patterns from Big Data using aggregation, filtering and tagging- A survey. In Proceedings of the 5th International Conference on Confluence 2014: The Next Generation Information Technology Summit: The Next Generation Information Technology Summit (pp. 66-71). [6949238] (Proceedings of the 5th International Conference on Confluence 2014: The Next Generation Information Technology Summit). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CONFLUENCE.2014.6949238
Prakashbhai, Pathak Anand ; Pandey, Hari Mohan. / Inference patterns from Big Data using aggregation, filtering and tagging- A survey. Proceedings of the 5th International Conference on Confluence 2014: The Next Generation Information Technology Summit: The Next Generation Information Technology Summit. Institute of Electrical and Electronics Engineers Inc., 2014. pp. 66-71 (Proceedings of the 5th International Conference on Confluence 2014: The Next Generation Information Technology Summit).
@inproceedings{b568073f01864887b480a8e54c3a3726,
title = "Inference patterns from Big Data using aggregation, filtering and tagging- A survey",
abstract = "This paper reviews various approaches to infer the patterns from Big Data using aggregation, filtering and tagging. Earlier research shows that data aggregation concerns about gathered data and how efficiently it can be utilized. It is understandable that at the time of data gathering one does not care much about whether the gathered data will be useful or not. Hence, filtering and tagging of the data are the crucial steps in collecting the relevant data to fulfill the need. Therefore the main goal of this paper is to present a detailed and comprehensive survey on different approaches. To make the concept clearer, we have provided a brief introduction of Big Data, how it works, working of two data aggregation tools (namely, flume and sqoop), data processing tools (hive and mahout) and various algorithms that can be useful to understand the topic. At last we have included comparisons between aggregation tools, processing tools as well as various algorithms through its pre-process, matching time, results and reviews.",
keywords = "aggregation, Big Data, filtering, Flume, Hadoop, HDFS, Hive, Mahout, MapReduce, Sqoop, tagging",
author = "Prakashbhai, {Pathak Anand} and Pandey, {Hari Mohan}",
year = "2014",
month = "11",
day = "6",
doi = "10.1109/CONFLUENCE.2014.6949238",
language = "English",
isbn = "9781479942367",
series = "Proceedings of the 5th International Conference on Confluence 2014: The Next Generation Information Technology Summit",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "66--71",
booktitle = "Proceedings of the 5th International Conference on Confluence 2014: The Next Generation Information Technology Summit",
address = "United States",

}

Prakashbhai, PA & Pandey, HM 2014, Inference patterns from Big Data using aggregation, filtering and tagging- A survey. in Proceedings of the 5th International Conference on Confluence 2014: The Next Generation Information Technology Summit: The Next Generation Information Technology Summit., 6949238, Proceedings of the 5th International Conference on Confluence 2014: The Next Generation Information Technology Summit, Institute of Electrical and Electronics Engineers Inc., pp. 66-71, 5th International Conference on Confluence 2014 - The Next Generation Information Technology Summit, Noida, India, 25/09/14. https://doi.org/10.1109/CONFLUENCE.2014.6949238

Inference patterns from Big Data using aggregation, filtering and tagging- A survey. / Prakashbhai, Pathak Anand; Pandey, Hari Mohan.

Proceedings of the 5th International Conference on Confluence 2014: The Next Generation Information Technology Summit: The Next Generation Information Technology Summit. Institute of Electrical and Electronics Engineers Inc., 2014. p. 66-71 6949238 (Proceedings of the 5th International Conference on Confluence 2014: The Next Generation Information Technology Summit).

Research output: Chapter in Book/Report/Conference proceedingConference proceeding (ISBN)

TY - GEN

T1 - Inference patterns from Big Data using aggregation, filtering and tagging- A survey

AU - Prakashbhai, Pathak Anand

AU - Pandey, Hari Mohan

PY - 2014/11/6

Y1 - 2014/11/6

N2 - This paper reviews various approaches to infer the patterns from Big Data using aggregation, filtering and tagging. Earlier research shows that data aggregation concerns about gathered data and how efficiently it can be utilized. It is understandable that at the time of data gathering one does not care much about whether the gathered data will be useful or not. Hence, filtering and tagging of the data are the crucial steps in collecting the relevant data to fulfill the need. Therefore the main goal of this paper is to present a detailed and comprehensive survey on different approaches. To make the concept clearer, we have provided a brief introduction of Big Data, how it works, working of two data aggregation tools (namely, flume and sqoop), data processing tools (hive and mahout) and various algorithms that can be useful to understand the topic. At last we have included comparisons between aggregation tools, processing tools as well as various algorithms through its pre-process, matching time, results and reviews.

AB - This paper reviews various approaches to infer the patterns from Big Data using aggregation, filtering and tagging. Earlier research shows that data aggregation concerns about gathered data and how efficiently it can be utilized. It is understandable that at the time of data gathering one does not care much about whether the gathered data will be useful or not. Hence, filtering and tagging of the data are the crucial steps in collecting the relevant data to fulfill the need. Therefore the main goal of this paper is to present a detailed and comprehensive survey on different approaches. To make the concept clearer, we have provided a brief introduction of Big Data, how it works, working of two data aggregation tools (namely, flume and sqoop), data processing tools (hive and mahout) and various algorithms that can be useful to understand the topic. At last we have included comparisons between aggregation tools, processing tools as well as various algorithms through its pre-process, matching time, results and reviews.

KW - aggregation

KW - Big Data

KW - filtering

KW - Flume

KW - Hadoop

KW - HDFS

KW - Hive

KW - Mahout

KW - MapReduce

KW - Sqoop

KW - tagging

UR - http://www.scopus.com/inward/record.url?scp=84914171236&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84914171236&partnerID=8YFLogxK

UR - http://www.mendeley.com/research/inference-patterns-big-data-using-aggregation-filtering-tagging-survey

U2 - 10.1109/CONFLUENCE.2014.6949238

DO - 10.1109/CONFLUENCE.2014.6949238

M3 - Conference proceeding (ISBN)

AN - SCOPUS:84914171236

SN - 9781479942367

T3 - Proceedings of the 5th International Conference on Confluence 2014: The Next Generation Information Technology Summit

SP - 66

EP - 71

BT - Proceedings of the 5th International Conference on Confluence 2014: The Next Generation Information Technology Summit

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Prakashbhai PA, Pandey HM. Inference patterns from Big Data using aggregation, filtering and tagging- A survey. In Proceedings of the 5th International Conference on Confluence 2014: The Next Generation Information Technology Summit: The Next Generation Information Technology Summit. Institute of Electrical and Electronics Engineers Inc. 2014. p. 66-71. 6949238. (Proceedings of the 5th International Conference on Confluence 2014: The Next Generation Information Technology Summit). https://doi.org/10.1109/CONFLUENCE.2014.6949238