OPI@LT-EDI-ACL2022: Detecting Signs of Depression from Social Media Text using RoBERTa Pre-trained Language Models

Abstract

This paper presents our winning solution for the Shared Task on Detecting Signs of Depression from Social Media Text at LT-EDI-ACL2022.The task was to create a system that, given social media posts in English, should detect the level of depression as 'not depressed', 'moderately depressed' or 'severely depressed'.We based our solution on transformer-based language models.We fine-tuned selected models: BERT, RoBERTa, XLNet, of which the best results were obtained for RoBERTa large .Then, using the prepared corpus, we trained our own language model called DepRoBERTa (RoBERTa for Depression Detection).Fine-tuning of this model improved the results.The third solution was to use the ensemble averaging, which turned out to be the best solution.It achieved a macro-averaged F1-score of 0.583.The source code of prepared solution is available at https://github.com/rafalposwiata/depression-detection-lt-edi-2022.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 57 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.