Problems and available solutions on the stage of Extract, Transform, and Loading in near real-time data warehousing (a literature study)

Abstract

In the traditional ETL (Extract Transform Loading), refreshment of data warehouse must be done in off peak hours. It means that all operational and analysis stopped from their all activities. It cause the level of freshness of data in the data warehouse is not indicating the latest operational transaction. This problem is called by data latency. Near real time data warehousing is used to be a solution for this problem. It update data warehouse in near real time manner, immediately after change data detected in data source. Thus, data latency can be minimized. In development, near real time data warehousing have problems where previously not found on the traditional ETL. This paper aims to convey the problems and available solutions at each stage in the near real time data warehousing, i.e. extraction, transformation, and loading. The problems and available solutions are based on literature review from other research that focusing on near real time data warehousing problem.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 26 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.