这里主要是整理自己在研究多模态嘲讽检测邻域时阅读到的比较好有借鉴意义的论文。
【论文阅读分类】
- 多模态嘲讽检测
- CLIP
- 对比学习
- 多模态命名实体识别
(带*为精读/代码复现)
多模态嘲讽检测
- *HFM(ACL 2019): Multi-Modal Sarcasm Detection in Twitter with Hierarchical Fusion Model
- D&R Net(ACL 2020): Reasoning with Multimodal Sarcastic Tweets via Modeling Cross-Modality Contrast and Semantic Association
- Rse-BERT/Att-BERT(EMNLP 2020): Modeling Intra and Inter-modality Incongruity for Multi-Modal Sarcasm Detection
- InCrossMGs(MM 2021): Multi-Modal Sarcasm Detection with Interactive In-Modal and Cross-Modal Graphs
- CMGCN(ACL 2022): Multi-Modal Sarcasm Detection via Cross-Modal Graph Convolutional Network
- *HKE(EMNLP 2022): Towards Multi-Modal Sarcasm Detection via Hierarchical Congruity Modeling with Knowledge Enhancement
(HKE代码运行时，由于显存一直在变化，故每次运行结果都不同，在此框架下加创新点并运行比较耗时，且容易出现显存不足的问题。所以最后还是选择了换框架 /(ㄒoㄒ)/~~) - *MILNet(AAAI 2023): Mutual-Enhanced Incongruity Learning Network for Multi-Modal Sarcasm Detection
- *DIP(CVPR 2023): DIP: Dual Incongruity Perceiving Network for Sarcasm Detection
CLIP
*CLIP(ICML 2021): Learning Transferable Visual Models From Natural Language Supervision
代码开源：https://github.com/openai/CLIP
API 调用：https://huggingface.co/docs/transformers/model_doc/clip
对比学习
- *CLNSN(NeurIPS 2021): Robust Contrastive Learning Using Negative Samples with Diminished Semantics
- SRCL(ACL 2023): Vision Language Pre-training by Contrastive Learning with Cross-Modal Similarity Regulation
- *NCLA(AAAI 2023): Neighbor Contrastive Learning on Learnable Graph Augmentation
