Tutorials
Prof. Itaru Kitahara
University of Tsukuba
Novel Viewpoint Synthesis in a Large-Scale Space for Live Free Viewpoint Sports Broadcast
Since sports broadcasting is a key content of visual media, various technologies for video processing, communication, and analysis have been developed to improve the visual quality of the broadcasts. This tutorial overviews the development of free viewpoint video generation and browsing methods from the dawn of research activities aimed at realizing live free viewpoint broadcast for large-scale spaces in the early 2000's, including the introduction of practical applications. Also, the possibility of camera work control using artificial intelligence and novel viewpoint synthesis technology based on deep learning are discussed.
Itaru Kitahara received his B.E. and M.E. degrees in Science Engineering from University of Tsukuba, Japan in 1994 and 1996, respectively. In 1996, he joined Sharp Corporation. 2000-2003, he was a research associate of University of Tsukuba. He received his Ph.D. in 2003. 2003-2005, he was a researcher at ATR. 2005-2019, he was an assistant professor and associate professor at the University of Tsukuba. Since 2019, he has been a professor at the Center for Computational Sciences, University of Tsukuba. He is also technical/academic advisor for IT companies. His research interests include computer vision, mixed reality, and intelligent image media.
[Education and Qualifications Obtained]
Doctor of Engineering (Ph.D.), University of Tsukuba (March 2003)
Master of Engineering, University of Tsukuba (March 1996)
Bachelor of Engineering, University of Tsukuba (March 1994)
[Work Experience]
- 2019-08 - (current) Professor, Center for Computational Sciences, University of Tsukuba & Center for Cybernics Research, University of Tsukuba
- 2023-04- (current) Academic Advisor, Space-Time Observatory Ltd.
- 2020-07 - (current) Board member, Research Institute for Connected Societies
- 2019-01- (current) Technical Advisor, Denqvision, Inc.
- 2016-04 - 2019-07 Associate Professor, Center for Computational Sciences, University of Tsukuba
- 2011-10 - 2016-03 Associate Professor, Faculty of Engineering, Information and Systems, University of Tsukuba Systems, University of Tsukuba
- 2005-04 - 2008-07 Assistant Professor, Graduate School of Systems and Information Engineering, University of Tsukuba
- 2003-04 - 2005-03 Researcher, Advanced Telecommunications Research Institute International(ATR)
- 2000-04 - 2003-03 Research Associate, Center for Tsukuba Advanced Research Alliance (TARA), University of Tsukuba
- 1996-04 - 1999-03 Sharp Corporation, Image Media Laboratory
Prof. Hansung Kim
School of Electronics & Computer Science, University of Southampton, UK
Immersive 3D Audio-Visual Room Reproduction in Virtual Reality
As personalized immersive display systems have been intensely explored in Virtual Reality (VR), plausible 3D audio corresponding to the visual content is required to provide more realistic experiences to users. In this talk, a complete pipeline to simultaneously reconstruct 3D geometry and acoustic properties of the environment from a panoramic (360⁰) image is introduced. A semantic scene reconstruction and completion method using a deep convolutional neural network has been developed to estimate the complete semantic scene geometry in order to adopt spatial audio reproduction to the scene.
Dr. Hansung Kim is an Associate Professor in the School of Electronics & Computer Science at the University of Southampton, UK. He received his BSc, MSc and Ph.D degrees in electronic and electrical engineering from Yonsei University, South Korea. He was employed as a researcher of Knowledge Science Lab (KSL) at Advanced Telecommunications Research Institute International (ATR), Japan, from 2005 to 2008, and as a senior research fellow at the Centre for Vision, Speech, and Signal Processing (CVSSP) at the University of Surrey, UK, from 2008 to 2020. His research for last 20 years includes 3D computer vision, Stereo image processing, 3D reconstruction, Augmented/Virtual Reality, Multi-modal data processing, Audio-visual data processing and Media production with over 100 published articles in 16 research projects.
Dr. Sungho Suh
German Research Center for Artificial Intelligence (DFKI)
Solving Sensor-Based Activity Recognition Problems Using Self-supervised Learning and Generative Models
Feature extraction is at the core of Human Activity Recognition (HAR), which involves automatically determining the activity being performed. Traditionally, the HAR community relied on statistical metrics and distribution-based representations to condense movement information from sensor data windows into feature vectors. In recent times, learned representations have emerged as effective alternatives to manually engineered features. Notably, self-supervised methods, harnessing large-scale unlabeled data for initial representation learning and subsequent fine-tuning for target applications, have piqued significant interest in the community. This tutorial focuses on representations for both single-sensor and multi-modal configurations, surpassing the current standard for representation learning. It also explores the efficient use of existing representations, especially through transfer learning and domain adaptation. The tutorial introduces cutting-edge methods for representation learning in HAR, providing a platform for researchers in mobile and ubiquitous computing to not only assess the field's current state but also to outline future directions, including the path to resolving the activity recognition challenge. Additionally, we will explore the integration of generative models to synthesize sensor data, further enhancing HAR performance.
Dr. Sungho Suh is a Senior Researcher at the German Research Center for Artificial Intelligence (DFKI) in Germany since 2021. He received the Ph.D. degree in Computer Science at the Technische Universität Kaiserslautern, Germany in 2021, and the B.S. and M.S. degrees from the School of Electrical Engineering and Computer Science, Seoul National University, Seoul, South Korea, in 2009 and 2011, respectively. Before joining DFKI, he worked at KIST Europe in Germany for three years, and at Samsung Electro-Mechanics, Korea from 2011 to 2018. His research interests are machine learning algorithms, such as sensor data processing, computer vision, multimodal processing, and generative models, with a focus on industrial applications.
Mr. Chaeun Lee
SAPEON Korea Inc.
Recent advances in quantization for deep learning models from algorithms to system level
To efficiently train and infer CNN-based deep neural networks in edge devices with limited resources, a range of model quantization techniques has been developed. Collecting data for such quantization, however, poses significant privacy challenges. To overcome this, data-free post-training quantization, which bypasses the need for the original data, has emerged as a solution. Additionally, the advancement in deep learning models extends to various tasks beyond traditional domain-specific recognition tasks such as vision and speech recognition introduces new and more complex model architectures. Particularly noteworthy are transformer-based large models known for their extensive parameter sets. However, these models often run into memory-bound limitations. Addressing these challenges, low-bit integer/floating-point quantization algorithms designed for these models and system-level optimization techniques have been proposed. This tutorial provides an overview of quantization algorithms and introduces emerging data-free quantization. It specifically addresses the application of different quantization algorithms to recent deep learning models and delves into system-level optimizations for transformer-based models, focusing on memory-bound issues.
Chaeun Lee received the B.S. and M.S. degrees in Electrical and Computer Engineering from Seoul National University (SNU) in 2018 and 2020, respectively. From 2020 to 2021, he worked as a researcher at Inter-university Semiconductor Research Center (ISRC) and Pohang University of Science and Technology (POSTECH). From 2022, he joined SAPEON Korea Inc. as a software engineer of algorithm team. His research topics are lightweight deep learning models, hardware-software co-optimization, and methodologies of simulation and emulation for lightweight models in customized devices.
