Spatial AI Lab – Zurich: Advancing AI in Computer Vision and Spatial Understanding

Spatial AI Lab – Zurich: Advancing the Frontiers of Artificial Intelligence

This document outlines the research focus and publications of the Spatial AI Lab at Microsoft Research in Zurich. The lab is dedicated to pushing the boundaries of artificial intelligence, with a particular emphasis on computer vision, spatial understanding, and their applications in areas like robotics, augmented reality, and virtual reality.

Research Areas and Focus

The Spatial AI Lab's research spans several key areas within AI and computer vision:

Computer Vision: This includes advancements in image and video analysis, object recognition, scene understanding, and 3D reconstruction.
Artificial Intelligence: A broad focus on developing intelligent systems, including machine learning algorithms, deep learning models, and their applications.
Spatial AI: Specializing in AI that understands and interacts with the physical world, leveraging 3D data and spatial reasoning.
3D Scene Understanding: Developing methods to interpret and analyze complex 3D environments, including object functionality and affordances.
Robotics: Research aimed at enabling robots to perceive, navigate, and interact with their surroundings intelligently.
Augmented and Virtual Reality: Creating immersive experiences and intelligent interfaces by integrating AI with spatial computing.

Key Publications and Contributions

The lab has a strong publication record, with numerous contributions to top-tier AI and computer vision conferences such as CVPR and ICCV. Some notable publications include:

SceneFun3D: Fine-Grained Functionality and Affordance Understanding in 3D Scenes: This work focuses on understanding the functional properties and affordances of objects within 3D scenes, a crucial step towards more intelligent scene interpretation.
LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometry: This research addresses visual odometry, a fundamental problem in robotics and AR/VR, by proposing a robust tracking method for accurate self-localization.
MuRF: Multi-Baseline Radiance Fields: This paper explores novel approaches to radiance fields, a powerful technique for novel view synthesis and 3D scene representation.
CASSPR: Cross Attention Single Scan Place Recognition: This work tackles the challenge of place recognition using single scans, important for autonomous navigation and mapping.
HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World: This publication introduces a valuable dataset for studying human-AI interaction in real-world scenarios, particularly relevant for assistive technologies.
Handbook on Leveraging Lines for Two-View Relative Pose Estimation: This provides a comprehensive guide to using geometric cues like lines for estimating the relative pose between cameras.
IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis: This research extends Neural Radiance Fields (NeRFs) to learn intrinsic scene properties, enabling more flexible scene editing and manipulation.
GlueStick: Robust Image Matching by Sticking Points and Lines Together: This paper presents a robust method for image matching by combining point and line features, essential for many computer vision tasks.
R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras: This work focuses on reconstructing dynamic 3D scenes from multiple camera views, enabling applications in motion capture and scene analysis.
RegFormer: An Efficient Projection-Aware Transformer Network for Large-Scale Point Cloud Registration: This research proposes an efficient transformer-based network for registering large-scale point clouds, critical for 3D mapping and alignment.

Leadership and People

The lab is led by Marc Pollefeys, a prominent figure in computer vision and AI, serving as the Partner Director of Science. The research team comprises numerous talented individuals, including:

Pablo Speciale
Ayc¸a Takmaz
Federico Tombari
Robert Sumner
Francis Engelmann
Weirong Chen
Le Chen
Rui Wang
Haofei Xu
Anpei Chen
Yuedong Chen
Christos Sakaridis
Yulun Zhang
Andreas Geiger
Fisher Yu
Yan Xia
Mariia Gladkova
Qianyun Li
Uwe Stilla
Joao F. Henriques
Daniel Cremers
Xin Wang
Taein Kwon
Mahdi Rad
Bowen Pan
Ishani Chakraborty
Sean Andrist
Dan Bohus
Ashley Feniello
Felipe Vieira Frujeri
Neel Joshi
Petr Hrubý
Shaohui Liu
Remi Pautrat
D. Baráth
Weicai Ye
Shuo Chen
Chong Bao
Hujun Bao
Zhaopeng Cui
Guofeng Zhang
Iago Suárez
Yifan Yu
Viktor Larsson
Aron Schmied
Tobias Fischer
Martin Danelljan
Jiuming Liu
Guangming Wang
Zhe Liu
Chaokang Jiang
Hesheng Wang

Location and Contact Information

The Spatial AI Lab is located at:

Seestrasse 356, Zurich 8038 Switzerland

Contact and social media links are provided, including connections to X (formerly Twitter), Facebook, LinkedIn, and YouTube, allowing for engagement with the lab's work and updates.

Further Resources

The page also includes links to various Microsoft resources, including product information, developer tools, educational initiatives, and company information, providing a comprehensive overview of Microsoft's engagement in technology and research.

This summary highlights the core activities and achievements of the Spatial AI Lab in Zurich, showcasing their significant contributions to the field of artificial intelligence and computer vision.