Big Data Architecture for Capturing, Storing, Analyzing and Visualizing of Web Server Logs

Abstract

Background/Objectives: To propose an Automated model to capture, store, analyze and visualize of Big Data. Methods/ Statistical Analysis: The web server logs are captured in real time and are stored into NoSQL database. The analysis on web server logs is performed using object-oriented language plus mongodb and finally the analyzed results are visualized in the R environment. Findings: To do a real time analysis of web server logs in data center using mongodb. Tracking of client location can be done using GeoIP. This GeoIP location information can be analyzed to find out the total number of visits from different country, real time information about server status of success, failure, warning and fatal errors. Real time updates of exact location of client who access sever and real time information of server status helpful to avoid repeating failure in a system. Application/Improvements: For the future work, we have planned to integrate our automated model with stream data processing system and to compare our proposed model with existing framework. Keywords: Analyzing, Big Data, Capturing, Storing, Visualizing, Web Server Logs

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 16 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.