Gerard de Melo's Publications
Publications: Computer Vision/Multimodality
See also Google Scholar and DBLP for some of the latest ones.
Publications by Topic: All | LLMs and NLP | Knowledge Graphs | Computer Vision/Multimodality | Social Media/Search | Source Code- Mitigate the Gap: Improving Cross-Modal Alignment in CLIP BibTeX Code
Sedigheh Eslami, Gerard de Melo (2025)
In: The Thirteenth International Conference on Learning Representations (ICLR 2025). - Vector Grimoire: Codebook-based Shape Generation under Raster Image Supervision BibTeX Website arXiv
Marco Cipriano, Moritz Feuerpfeil, Gerard de Melo (2025)
In: Proc. ICML 2025. - MaskAnyone Toolkit: Offering Strategies for Minimizing Privacy Risks and Maximizing Utility in Audio-Visual Data Archiving BibTeX
Babajide Alamu Owoyele, Martin Schilling, Rohan Sawahn, Niklas Kaemer, Pavel Zherebenkov, Bhuvanesh Verma, Wim Pouw, Gerard de Melo (2024)
In: Proc. ICIS 2024. Association for Information Systems. - On the Challenges and Opportunities in Generative AI BibTeX
Laura Manduchi, Kushagra Pandey, Robert Bamler, Ryan Cotterell, Sina Däubener, Sophie Fellenz, Asja Fischer, Thomas Gärtner, Matthias Kirchler, Marius Kloft, Yingzhen Li, Christoph Lippert, Gerard de Melo, Eric Nalisnick, Björn Ommer, Rajesh Ranganath, Maja Rudolph, Karen Ullrich, Guy Van den Broeck, Julia E Vogt, Yixin Wang, Florian Wenzel, Frank Wood, Stephan Mandt, Vincent Fortuin (2024)
ArXiv 2403.00025, 2024.ArtQuest: Countering Hidden Language Biases in ArtVQA BibTeX Presentation Video
Tibor Bleidt, Sedigheh Eslami, Gerard de Melo (2024)
In: Proc. IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2024).Wiki-VEL: Visual Entity Linking for Structured Data on Wikimedia Commons BibTeX
Philipp Bielefeld, Jasmin Geppert, Necdet Güven, Melna John, Adrian Ziupka, Lucie-Aimée Kaffee, Russa Biswas, Gerard De Melo (2024)
In: Proc. 3rd Workshop on Advances in Language and Vision Research (ALVR). Association for Computational Linguistics.Multi-Modal Bias: Introducing a Framework for Stereotypical Bias Assessment beyond Gender and Race in Vision–Language Models BibTeX arXiv Presentation Video
Sepehr Janghorbani, Gerard de Melo (2023)
In: Proc. EACL 2023. Association for Computational Linguistics.PubMedCLIP: How Much Does CLIP Benefit Visual Question Answering in the Medical Domain? BibTeX arXiv Presentation Video Data Code
Sedigheh Eslami, Christoph Meinel, Gerard de Melo (2023)
In: Findings of the Association for Computational Linguistics: EACL 2023. Association for Computational Linguistics.ViLPAct: A Benchmark for Compositional Generalization on Multimodal Human Activities BibTeX arXiv Presentation Video
Terry Yue Zhuo, Yaqing Liao, Yuecheng Lei, Lizhen Qu, Gerard de Melo, Xiaojun Chang, Yazhou Ren, Zenglin Xu (2023)
In: Findings of the Association for Computational Linguistics: EACL 2023. Association for Computational Linguistics.FARSEC: A Reproducible Framework for Automatic Real-Time Vehicle Speed Estimation Using Traffic Cameras BibTeX
Lucas Liebe, Franz Sauerwald, Sylwester Sawicki, Matthias Schneider, Leo Schuhmann, Tolga Buz, Paul Boes, Ahmad Ahmadov, Gerard de Melo (2023)
ArXiv 2309.14468, 2023.Purely Attention Based Local Feature Integration for Video Classification BibTeX
Xiang Long, Gerard de Melo, Dongliang He, Fu Li, Zhizhen Chi, Shilei Wen, Chuang Gan (2022)
IEEE Transactions on Pattern Analysis and Machine Intelligence 44:4, 2022, p. 2140–2154.
Impact factor: 17.861Purely Attention Based Local Feature Integration for Video Classification BibTeX
Xiang Long, Gerard de Melo, Dongliang He, Fu Li, Zhizhen Chi, Shilei Wen, Chuang Gan (2022)
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 44:4, 2022, p. 2140–2154.
Impact factor: 17.861 (2019)Improving Personalized Explanation Generation through Visualization BibTeX
Shijie Geng, Zuohui Fu, Yingqiang Ge, Lei Li, Gerard de Melo, Yongfeng Zhang (2022)
In: Proc. ACL 2022. Association for Computational Linguistics.Art Creation with Multi-Conditional StyleGANs BibTeX arXiv Presentation Video Code
Konstantin Dobler, Florian Hübscher, Jan Westphal, Alejandro Sierra-Múnera, Gerard de Melo, Ralf Krestel (2022)
In: Proc. IJCAI-ECAI 2022 (Special Track on AI, the Arts and Creativity).Frozen CLIP Models are Efficient Video Learners BibTeX arXiv Code
Ziyi Lin, Shijie Geng, Renrui Zhang, Peng Gao, Gerard de Melo, Xiaogang Wang, Jifeng Dai, Yu Qiao, Hongsheng Li (2022)
In: Proc. ECCV 2022. Springer Lecture Notes in Computer Science.Masked-Piper: Masking personal identities in visual recordings while preserving multimodal information BibTeX Code
Babajide Owoyele, James Trujillo, Gerard de Melo, Wim Pouw (2022)
SoftwareX, 2022. Elsevier.
Impact factor: 3.4 (2022)TIME: Text and Image Mutual-Translation Adversarial Networks BibTeX arXiv Slides
Bingchen Liu, Kunpeng Song, Yizhe Zhu, Gerard de Melo, Ahmed Elgammal (2021)
In: Proc. AAAI 2021.
Acceptance rate: 21%Dense Contrastive Visual-Linguistic Pretraining BibTeX arXiv Code
Lei Shi, Kai Shuang, Shijie Geng, Peng Su, Zhengkai Jiang, Peng Gao, Zuohui Fu, Gerard de Melo, Sen Su (2021)
In: Proc. ACM Multimedia (MM) 2021.
Acceptance rate: ~27.9%Semantics-Aware Typographical Choices via Affective Associations BibTeX
Tugba Kulahcioglu, Gerard de Melo (2021)
Language Resources and Evaluation 55:1, 2021, p. 105–126. Springer Verlag.Exploiting Image–Text Synergy for Contextual Image Captioning BibTeX Poster Code
Sreyasi Nag Chowdhury, Rajarshi Bhowmik, Hareesh Ravi, Gerard de Melo, Simon Razniewski, Gerhard Weikum (2021)
In: Proc. 3rd Workshop Beyond Vision and Language: Integrating Real-world Knowledge (LANTERN), colocated with EACL 2021. ACL.
🏆 Best Paper AwardTeamS at VQA-Med 2021: BBN-Orchestra for Long-tailed Medical Visual Question Answering BibTeX
Sedigheh Eslami, Gerard de Melo, Christoph Meinel (2021)
In: CLEF 2021 Working Notes. CEUR-WS CEUR Workshop Proceedings.
🏆 Ranked 3rd in ImageCLEF VQA-Med 2021Incorporating Pragmatic Reasoning Communication into Emergent Language BibTeX arXiv Presentation Video
Yipeng Kang, Tonghan Wang, Gerard de Melo (2020)
In: Proc. NeurIPS 2020 (Spotlight Paper).
Acceptance rate: 20% (4% for spotlight/oral)OOGAN: Disentangling GAN with One-Hot Sampling and Orthogonal Regularization BibTeX Website/Code arXiv
Bingchen Liu, Yizhe Zhu, Zuohui Fu, Gerard de Melo, Ahmed Elgammal (2020)
In: Proc. AAAI 2020. AAAI Press.
Acceptance rate: 20.6%Long Short-Term Sample Distillation BibTeX arXiv
Liang Jiang, Zujie Wen, Zhongping Liang, Yafang Wang, Gerard de Melo, Zhe Li, Liangzhuang Ma, Jiaxing Zhang, Xiaolong Li, Yuan Qi (2020)
In: Proc. AAAI 2020. AAAI Press.
Acceptance rate: 20.6%SCALOR: Generative World Models with Scalable Object Representations BibTeX Website arXiv Slides Code
Jindong Jiang, Sepehr Janghorbani, Gerard de Melo, Sungjin Ahn (2020)
In: Proc. ICLR 2020. OpenReview.net.
Acceptance rate: 26.5%Fonts Like This but Happier: A New Way to Discover Fonts BibTeX
Tugba Kulahcioglu, Gerard de Melo (2020)
In: Proc. ACM Multimedia 2020. ACM.
Acceptance rate: 27.8%EmoTag1200 👍: Understanding the Association between Emojis 😄 and Emotions 😻 BibTeX Website arXiv Slides Presentation Video
Abu Awal Md Shoeb, Gerard de Melo (2020)
In: Proc. EMNLP 2020. Association for Computational Linguistics.
Acceptance rate: 24.6%Affect-Aware Word Clouds BibTeX
Tugba Kulahcioglu, Gerard de Melo (2020)
ACM Transactions on Interactive Intelligent Systems (TiiS), 2020. ACM.
Impact factor: 1.63 (2020)Character Matters: Video Story Understanding with Character-Aware Relations BibTeX
Shijie Geng, Ji Zhang, Zuohui Fu, Peng Gao, Hang Zhang, Gerard de Melo (2020)
ArXiv 2005.08646, 2020.Employing Shadows for Multi-Person Tracking Based on a Single RGB-D Camera BibTeX
Wei Gai, Meng Qi, Mingcong Ma, Lu Wang, Chenglei Yang, Juan Liu, Yulong Bian, Gerard de Melo, Shijun Liu, Xiangxu Meng (2020)
Sensors 20:4, 2020, p. 1056. MDPI.Illustrate Your Story: Enriching Text with Images BibTeX Presentation Video
Sreyasi Nag Chowdhury, William Cheng, Gerard de Melo, Simon Razniewski, Gerhard Weikum (2020)
In: Proc. WSDM 2020 (Demonstration Paper). ACM.MR Environments Constructed for a Large Indoor Physical Space BibTeX Presentation Video
Huan Xing, Chenglei Yang, Xiyu Bao, Sheng Li, Wei Gai, Meng Qi, Juan Liu, Yuliang Shi, Gerard de Melo, Fan Zhang, Xiangxu Meng (2020)
In: CGI 2020: Advances in Computer Graphics. Springer International Publishing.Paralinguistic Recommendations for Affective Word Clouds BibTeX Presentation Video
Tugba Kulahcioglu, Gerard de Melo (2019)
In: Proc. ACM IUI 2019. ACM.
Acceptance rate: 25%EmoTag – Towards an Emotion-Based Analysis of Emojis BibTeX Website Slides
Abu Awal Md Shoeb, Shahab Raji, Gerard de Melo (2019)
In: Proc. RANLP 2019.
Acceptance rate: 8.7% (Long Presentation Papers)CITE: A Corpus Of Text-Image Discourse Relations BibTeX Website/Code arXiv
Malihe Alikhani, Sreyasi Nag Chowdhury, Gerard de Melo, Matthew Stone (2019)
In: Proc. NAACL-HLT 2019 (Short Paper). Association for Computational Linguistics.
Acceptance rate: 21.3%Leveraging Blowing as a Directly Controlled Interface BibTeX
Yeqing Chen, Yulong Bian, Chenglei Yang, Xiyu Bao, Yafang Wang, Gerard de Melo (2019)
In: Proc. IEEE SmartWorld 2019 (Short Paper). IEEE.Catch the Shadow: Person Tracking Under Occlusion with a Single RGB-D Camera BibTeX
Wei Gai, Meng Qi, Lu Wang, Chenglei Yang, Mingcong Ma, Juan Liu, Yulong Bian, Gerard de Melo, Shijun Liu, Xiangxu Meng (2019)
In: Proc. IEEE SmartWorld 2019 (Short Paper). IEEE.Optimization Strategies for Real-Time Rendering of Virtual Scenes on Heterogeneous Mobile Devices BibTeX
Wei Gai, Xiyu Bao, Meng Qi, Yafang Wang, Juan Liu, Gerard de Melo, Lu Wang, Lizhen Cui, Chenglei Yang, Xiangxu Meng (2019)
In: Proc. IEEE SmartWorld 2019 (Visual Perception and Visual Computing Paper). IEEE.Rotbav: A Toolkit for Constructing Mixed Reality Apps with Real-Time Roaming in Large Indoor Physical Spaces BibTeX
Huan Xing, Xiyu Bao, Fan Zhang, Wei Gai, Meng Qi, Juan Liu, Yuliang Shi, Gerard de Melo, Chenglei Yang, Xiangxu Meng (2019)
In: Proc. IEEE VR 2019 (Poster Paper). IEEE.Video Captioning with Multi-Faceted Attention BibTeX arXiv
Xiang Long, Chuang Gan, Gerard de Melo (2018)
Transactions of the Association for Computational Linguistics (TACL) 6, 2018, p. 173–184.
Acceptance rate: 19%Attention Clusters: Purely Attention Based Local Feature Integration for Video Classification BibTeX arXiv Supplemental Material Data
Xiang Long, Chuang Gan, Gerard de Melo, Jiajun Wu, Xiao Liu, Shilei Wen (2018)
In: Proc. CVPR 2018.
🏆 Backbone of winning solution in ActivityNet 2017 Kinetics Challenge
Acceptance rate: 29.6%Multimodal Keyless Attention Fusion for Video Classification BibTeX
Xiang Long, Chuang Gan, Gerard de Melo, Xiao Liu, Yandong Li, Fu Li, Shilei Wen (2018)
In: Proc. AAAI 2018. AAAI Press.
🏆 Backbone of 3rd place ($20,000 prize) solution in the Google Cloud & YouTube-8M Video Understanding Challenge
Acceptance rate: 25%Predicting Semantic Signatures of Fonts BibTeX Website
Tugba Kulahcioglu, Gerard de Melo (2018)
In: Proc. 12th IEEE International Conference on Semantic Computing.
🏆 Best Paper NomineeFontLex: A Typographical Lexicon based on Affective Associations BibTeX Website
Tugba Kulahcioglu, Gerard de Melo (2018)
In: Proc. 11th Language Resources and Evaluation Conference (LREC 2018). European Language Resources Association (ELRA).Multimodal Question Answering over Structured Data with Ambiguous Entities BibTeX
Huadong Li, Yafang Wang, Gerard de Melo, Changhe Tu, Baoquan Chen (2017)
In: Proc. WWW 2017 (Cognitive Computing Track). ACM.Concepts Not Alone: Exploring Pairwise Relationships for Zero-Shot Video Activity Recognition BibTeX
Chuang Gan, Ming Lin, Yi Yang, Gerard de Melo, Alexander G. Hauptmann (2016)
In: Proc. AAAI 2016. AAAI Press.
Acceptance rate: 26%ShapeLearner: Towards Shape-Based Visual Knowledge Harvesting BibTeX Website
Huayong Xu, Yafang Wang, Kang Feng, Gerard de Melo, Wei Wu, Andrei Sharf, Baoquan Chen (2016)
In: Proc. ECAI 2016. IOS Press Frontiers in Artificial Intelligence and Applications.
Acceptance rate: 27%Seeing is Believing: The Quest for Multimodal Knowledge BibTeX
Gerard de Melo, Niket Tandon (2016)
ACM SIGWEB Newsletter Spring 2016, 2016, p. 4:1–4:9. ACM.ShapeExplorer: Querying and Exploring Shapes using Visual Knowledge BibTeX Website
Tong Ge, Yafang Wang, Gerard de Melo, Zengguang Hao, Andrei Sharf, Baoquan Chen (2016)
In: Proc. EDBT 2016 (Demonstration Paper).Visualizing and Curating Knowledge Graphs over Time and Space BibTeX Presentation Video
Tong Ge, Yafang Wang, Gerard de Melo, Haofeng Li (2016)
In: Proc. ACL 2016 (Demonstration Paper). ACL.Knowlywood: Mining Activity Knowledge from Hollywood Narratives BibTeX Website Slides
Niket Tandon, Gerard de Melo, Abir De, Gerhard Weikum (2015)
In: Proc. CIKM 2015. ACM.
Acceptance rate: 18% (KR Track)Perceptually Grounded Selectional Preferences BibTeX Poster
Ekaterina Shutova, Niket Tandon, Gerard de Melo (2015)
In: Proc. ACL 2015.
Acceptance rate: 25%Lights, Camera, Action: Knowledge Extraction from Movie Scripts BibTeX Data
Niket Tandon, Gerard de Melo, Abir De, Gerhard Weikum (2015)
In: Proc. WWW 2015 (Poster Paper).
Acceptance rate: 30%
