Mei, Zhengkun (2023) Chinese Multi-Model Sarcasm Detection Based on Contrastive Attention Residual Late Fusion. Master thesis, Voice Technology (VT).
|
PDF
Mei_correction.pdf Download (443kB) | Preview |
Abstract
Sarcasm detection has become increasingly crucial in the era of virtual assistants and the widespread use of sarcasm on the internet. While previous research has focused on text, the significance of other modalities, such as audio and image, has gained prominence. However, the detection of sarcasm in the Chinese language remains limited to text-based approaches due to the lack of multimodal datasets. This paper addresses this research gap by introducing a novel Chinese multimodal sarcasm dataset that combines text and audio modalities. To effectively capture the rich information from different modalities, I propose a late fusion contrastive attention residual model. This model leverages convolutional fusion layers to fuse raw and high-dimensional features from multiple modalities, and the two modalities' information is organically combined through the mechanism of contrastive attention, facilitating the detection of sarcasm. The experimental results demonstrate the effectiveness of the proposed approach, showcasing improved performance in Chinese sarcasm detection. The detail result you can check in my paper. This research and newly created Chinese multi-model dataset contributes to advancing multimodal sarcasm detection techniques and lays the foundation for future studies in this domain. The full dataset is publicly available for use and academic research, and the Github link is https://github.com/ZhengkunMei/Chinese-multimodel-sarcasm-dataset
| Item Type: | Thesis (Master) |
|---|---|
| Name supervisor: | Coler, M.L. and Gao, X. |
| Date Deposited: | 12 Sep 2023 11:10 |
| Last Modified: | 12 Sep 2023 11:10 |
| URI: | https://campus-fryslan.studenttheses.ub.rug.nl/id/eprint/370 |
Actions (login required)
 |
View Item |