Investigating Dynamic Mutation Process of Issues Using Unstructured Text Analysis

Abstract

최근 가용한 텍스트 데이터 자원이 증가함에 따라 방대한 텍스트 분석을 통해 새로운 가치를 창출하고자 하는 수요가 증가하고 있다. 특히 뉴스, 민원, 블로그, SNS 등을 통해 유통되는 글로부터 다양한 이슈를 발굴해내고 이들 이슈의 추이를 분석하는 이슈 트래킹에 대한 연구가 활발하게 이루어지고 있다. 전통적인 이슈 트래킹은 토픽 모델링을 통해 오랜 기간에 걸쳐 지속된 주요 이슈를 발굴한 후, 각 이슈를 구성하는 문서 수의 세부 기간별 분포를 분석하는 방식으로 이루어진다. 하지만 전통적 이슈 트래킹은 각 이슈를 구성하는 내용이 전체 기간에 걸쳐 변화 없이 유지된다는 가정 하에 수행되기 때문에, 다양한 세부 이슈가 서로 영향을 주며 생성, 병합, 분화, 소멸하는 이슈의 동적 변이과정을 나타내지 못한다. 또한 전체 기간에 걸쳐 지속적으로 출현한 키워드만이 이슈 키워드로 도출되기 때문에, 핵실험, 이산가족 등 세부 기간의 분석에서는 매우 상이한 맥락으로 파악되는 구체적인 이슈가 오랜 기간의 분석에서는 북한이라는 큰 이슈에 함몰되어 가려지는 현상이 발생할 수 있다. 본 연구에서는 이러한 한계를 극복하기 위해 각 세부 기간의 문서에 대한 독립적인 분석을 통해 세부 기간별 주요 이슈를 도출한 후, 각 이슈의 유사도에 기반하여 이슈 흐름도를 도출하고자 한다. 또한 각 문서의 카테고리 정보를 활용하여 카테고리간의 이슈 전이 패턴을 분석하고자 한다. 본 논문에서는 총 53,739건의 신문 기사에 제안 방법론을 적용한 실험을 수행하였으며, 이를 통해 전통적인 이슈 트래킹을 통해 발굴한 주요 이슈의 세부 기간별 구성 내용을 살펴볼 수 있을 뿐 아니라, 특정 이슈의 선행 이슈와 후행 이슈를 파악할 수 있음을 확인하였다. 또한 카테고리간 분석을 통해 단방향 전이와 양방향 전이의 흥미로운 패턴을 발견하였다. Owing to the extensive use of Web media and the development of the IT industry, a large amount of data has been generated, shared, and stored. Nowadays, various types of unstructured data such as image, sound, video, and text are distributed through Web media. Therefore, many attempts have been made in recent years to discover new value through an analysis of these unstructured data. Among these types of unstructured data, text is recognized as the most representative method for users to express and share their opinions on the Web. In this sense, demand for obtaining new insights through text analysis is steadily increasing. Accordingly, text mining is increasingly being used for different purposes in various fields. In particular, issue tracking is being widely studied not only in the academic world but also in industries because it can be used to extract various issues from text such as news, (SocialNetworkServices) to analyze the trends of these issues. Conventionally, issue tracking is used to identify major issues sustained over a long period of time through topic modeling and to analyze the detailed distribution of documents involved in each issue. However, because conventional issue tracking assumes that the content composing each issue does not change throughout the entire tracking period, it cannot represent the dynamic mutation process of detailed issues that can be created, merged, divided, and deleted between these periods. Moreover, because only keywords that appear consistently throughout the entire period can be derived as issue keywords, concrete issue keywords such as "nuclear test" and "separated families" may be concealed by more general issue keywords such as "North Korea" in an analysis over a long period of time. This implies that many meaningful but short-lived issues cannot be discovered by conventional issue tracking. Note that detailed keywords are preferable to general keywords because the former can be clues for providing actionable strategies. To overcome these limitations, we performed an independent analysis on the documents of each detailed period. We generated an issue flow diagram based on the similarity of each issue between two consecutive periods. The issue transition pattern among categories was analyzed by using the category information of each document. In this study, we then applied the proposed methodology to a real case of 53,739 news articles. We derived an issue flow diagram from the articles. We then proposed the following useful application scenarios for the issue flow diagram presented in the experiment section. First, we can identify an issue that actively appears during a certain period and promptly disappears in the next period. Second, the preceding and following issues of a particular issue can be easily discovered from the issue flow diagram. This implies that our methodology can be used to discover the association between inter-period issues. Finally, an interesting pattern of one-way and two-way transitions was discovered by analyzing the transition patterns of issues through category analysis. Thus, we discovered that a pair of mutually similar categories induces two-way transitions. In contrast, one-way transitions can be recognized as an indicator that issues in a certain category tend to be influenced by other issues in another category. For practical application of the proposed methodology, high-quality word and stop word dictionaries need to be constructed. In addition, not only the number of documents but also additional meta-information such as the read counts, written time, and comments of documents should be analyzed. A rigorous performance evaluation or validation of the proposed methodology should be performed in future works.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 6 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.