Emotionwave director Park Seong Hyeon joined researchers from Sungkyunkwan University’s Department of AI Convergence in the study “Auditing Multimodal Claims:…
Proposed Audit and Verification Framework
Emotionwave director Park Seong Hyeon joined researchers from Sungkyunkwan University’s Department of AI Convergence in the study “Auditing Multimodal Claims: A Controlled Reproducibility Study of Graph-Based Conversational Recommendation,” which examined the multimodal performance of graph-based conversational recommender systems (CRS).
The study investigated how much visual information, such as images, actually contributes to recommendation performance. Keeping the model structure and training environment unchanged, the researchers used a Distractor Audit that replaced image embeddings with random, zero and shuffled-real vectors to measure the contribution of the visual modality.
Recommendation performance changed only modestly when visual information was disrupted, while adding LLM-generated synthetic dialogue data produced clear improvements. The results suggest that improvements in existing multimodal CRS may depend more on dialogue augmentation and changes in data composition than on the visual information itself.
Park also identified evaluation considerations, including missing data and checkpoint selection, and proposed a reproducibility audit protocol for assessing the actual contribution of each modality.
The study was accepted to the Main Track of CIKM 2026, the 35th ACM International Conference on Information and Knowledge Management, an international conference on information retrieval and data mining.
