In short, I develop computational models of human language. My scientific interests include modelling natural language processing, understanding, and generation, how human languages interact with our visual perceptions and representations, and how these interact with and are grounded by our commonsense and the external world.
I am a visiting post-doctoral research fellow in the Center for Data Science in New York University where I work with Kyunghyun Cho. I am also a post-doctoral research fellow in the Institute for Logic, Language and Computation in the University of Amsterdam, where I am affiliated to the Dialogue Modelling Group led by Raquel Fernández where I collaborate with many outstanding researchers working at the intersection of language & vision. Before that I worked with Khalil Sima’an's group on how to integrate visual information into different natural language processing tasks.
Some broad research fields I consider myself a contributor to include machine learning and representation learning for language and vision.
You can find me in
office 637, 60 5th Avenue, Manhattan online/by email!
2019-2022 IMAGINE: Integrating multimodal, world and commonsense knowledge into language generation. EU H2020 MSCA
2019-2023 Multi3Generation: Multi-task, Multilingual, Multimodal Language Generation. EU COST Action
Sep/2021 Collaborative article Neural Natural Language Generation: A Survey on Multilinguality, Multimodality, Controllability and Learning accepted for publication in the Journal of Artificial Intelligence Research (JAIR) where we discuss the recent trends in neural approaches to natural language generation. More news soon!
May/2021 Book chapter Natural language processing for mental disorders: an overview accepted for publication in: Natural Language Processing in Healthcare: A Special Focus on Low Resource Languages (CRC Press). More news soon!
May/2021 Our paper Seeing past words: Testing the cross-modal capabilities of pretrained V&L models on counting tasks was accepted for publication at the ‘Beyond Language: Multimodal Semantic Representations’ (MMSR 2021) Workshop. We release a counting probe dataset to be used in evaluating pretrained V&L models (data).
Mar/2021 Paper accepted at NAACL 2021: Wikipedia Entities as Rendezvous across Languages: Grounding Multilingual Language Models by Predicting Wikipedia Hyperlinks (code and data coming soon).
Feb/2021 I am guest lecturer in the Erasmus Mundus European Masters on Technology for Translation and Interpreting. I give lectures on Machine Translation and Machine Learning in the University of Wolverhampton.
Jan/2021 We published a pre-print on probing the counting capabilities of pretrained vision & language models: Seeing past words: Testing the cross-modal capabilities of pretrained V&L models?
Sep/2020 We release the beta-version of the VisualSem knowledge graph, a multimodal and multilingual resource to support vision & language research: VisualSem: a high-quality knowledge graph for vision and language (code)
Sep/2020 Two papers accepted at AACL 2020: Are scene graphs good enough to improve Image Captioning? and English intermediate-task training improves zero-shot cross-lingual transfer too.
Aug/2020 I am co-organizer of the Repl4NLP 2021 (Representation Learning for NLP 2021) workshop.
Jul/2020 I am Area Chair for Language Grounding to Vision at EACL 2021.
Aug/2019 I am Short-Term Scientific Mission Coordinator and member of the Management Committee of the Multi3Generation COST Action.
Jun/2019 I have joined Kyunghyun Cho’s group in New York University.
May/2019 Paper accepted at ACL 2019 where we propose a latent variable model for multi-modal machine translation (pre-print available here and code coming soon).
Mar/2019 I have joined Raquel Fernández’s group in the University of Amsterdam.
Feb/2019 I have received a Marie Skłodowska-Curie Global Fellowship! I will be visiting the New York University Centre for Data Science, Facebook Artificial Intelligence Research in Paris, just to return to the University of Amsterdam! My project is called IMAGINE: Improving lAnguage Generation wIth world kNowledgE and more information is available here.
Jun/2018 I gave two lectures at the Summer School in Translation Technologies organised by the Faculty of Arts in Universidade do Porto. The lectures were on statistical and neural machine translation.
Mar/2018 I have joined Khalil Sima’an’s group in the University of Amsterdam!
Oct/2017 Best Long Paper Award at INLG 2017 in a collaboration with Thiago Castro Ferreira, Sander Wubben and Emiel Krahmer with paper Linguistic realisation as machine translation: Comparing different MT models for AMR-to-text generation.