An introduction to multimodal learning and its applications

：董性平（武汉大学）
：2023-12-15 15:00
：海韵园数理大楼686会议室

报告人：董性平（武汉大学）

时  间：2023年12月15日15:00

地  点：海韵园数理大楼686会议室

内容摘要:

With the vigorous development of deep learning, many artificial intelligence algorithms have achieved tremendous success in machine learning tasks, such as speech recognition, machine translation, and facial recognition. However, most of these tasks are based on single-modal data, while real-world data is often multimodal, as seen in movie data that includes video, audio, and text subtitles. The development of multimodal learning not only enables the effective utilization of the growing multimodal data but also provides a more comprehensive understanding of real-world data, enhancing the efficiency and capabilities of models. It can also handle more complex tasks, such as the recently highlighted text-to-image generation task.

This presentation will briefly introduce the basic concepts and development of multimodal learning, along with its applications in the visual and language domains. Through the analysis of specific tasks, including visual language navigation, language referring object segmentation and tracking, we will explore the challenges and solutions that multimodal learning faces in practical applications, providing initial insights for the practical implementation of multimodal learning algorithms.

个人简介：

董性平，武汉大学计算机学院教授、博士生导师、国家级青年人才，曾任阿联酋起源人工智能研究院研究员。目前主要从事小样本学习、目标跟踪、图像/视频目标分割、视觉语言导航、深度强化学习、神经渲染、自动驾驶感知等的研究工作。2012年在厦门大学数学科学学院获得计算数学学士学位；2019年在北京理工大学计算机学院智能信息技术北京市重点实验室，获得计算机博士学位，师从沈建冰教授（IEEE Fellow）；同时为澳洲国立大学工程与计算机科学学院联合培养博士，师从国际著名视觉学者 Fatih Porikli 教授（IEEE Fellow）。曾获得中国人工智能学会优秀博士论文奖、北京理工大学优秀博士论文奖；已发表高水平论文 20 余篇，其中包括国际权威期刊IEEE Trans.汇刊 (IEEE TPAMI, IEEE TIP, IEEE TMM, IEEE TNNLS, IEEE TCYB, IEEE TCSVT) 和国际顶级视觉会议 (IEEE CVPR, ECCV)；2021-2023连续三年入选斯坦福大学全球前2％顶尖科学家年度影响力榜单；担任众多国际著名会议(IEEE CVPR, IEEE ICCV, ECCV, NeurIPS, ICLR, AAAI, ACCV, WACV) 和国际权威期刊 (IEEE TPAMI, IEEE TIP, IEEE TMM, IEEE TNNLS, IEEE TCSVT, IEEE TITS, PR, CVIU, NEUCOM, TVCJ, SIVP) 的常规审稿人。

 

联系人：谭志裕