Cross-Modal and Cross-Lingual Information Search and Retrieval.

This project focuses on extending and combining the multi-idiomatic and multi-modal settings to highly noisy and unstructured data, as naturally found on the Web (e.g., on social media, e-commerce sites, blog posts, etc.). I propose to develop novel multi-modal, multi-lingual and multi-idomatic retrieval and search models, grounded on the successful deep learning framework, to create innovative technology that jointly performs text analysis and computer vision tasks preserving the natural alignment between these two modalities, and thus aiming to close the semantic gap between them. Given my previous experience and the existing collaborations of our group with the Computer Vision group at ESAT and our active participation in the broader expert network in similar research tasks, I am condent that the goals stated above can be achieved in one year.