Abstract

Matching images to text plays an important role in cross-media retrieval and research has proven this to be an underestimated challenge. This problem is addressed by the MediaEval 2021 News-Images Challenge with the goal to gain more insights into the real-world relationship of news articles and images. We develop models for reestablishing the connection of an news article to its corresponding image using datasets of a German news publisher (task 1). Our approaches follows the idea of pairwise similarity learning and are optimized by algorithmic hill climbing. Additionally, we employ Canonical Correlation Analysis as an approach using joint embedding learning. The evaluation shows that our approaches produce good results for the underlying image-text rematching task, yet require further optimization to yield stable prediction performance.

@inproceedings{AbdulEtAl:RematchingUsingEmbeddingsAndCnns2021,
	author    = {Kani Abdul and Kiran Kiran and Max Rudat and Alexandros Vasileiou and Andreas Lommatzsch},
	title     = {Methods for Text-Image-Rematching using Pair-wise Similarity and Canonical Similarity Analysis},
	booktitle = {Proceedings of the MediaEval Benchmarking Initiative for Multimedia Evaluation 2021},
	year      = {2021},
	issn      = {1613-0073},
	url       = {https://ceur-ws.org/Vol-3181/paper45.pdf},
	location  = {Online},
	publisher = {CEUR Workshop Proceedings},
	keywords  = {workshop, paper, multi-media, nlp}
}