Speaker：Andrew Luo, Assistant Professor, University of Hong Kong
罢颈尘别： Dec. 23, 1:30--3:00 p.m.
痴别苍耻别：Room 1113，Wangkezhen Building
贬辞蝉迟：Prof. Fang Fang
Abstract
A fundamental goal of cognitive neuroscience has been understanding how the human visual cortex supports perceiving and interpreting visual information in the world around us. Traditional approaches to mapping the visual cortex have relied on manually assembled stimulus sets, often employing isolated objects in artificial contexts with simplified backgrounds. These approaches do not fully capture the complexity and richness of real-world visual experience, potentially biasing results and limiting our understanding of visual processing. I introduce a suite of computational approaches leveraging naturalistic image stimuli to identify and characterize the high-level organization of visual information in the human brain.
Specifically, I present:
1. Brain Diffusion for Visual Exploration (BrainDiVE): A method utilizing gradient guidance from a differentiable image-to-fMRI encoder and a pre-trained image diffusion model to generate naturalistic “most-exciting-inputs” that maximally activate specific brain regions.
2. Semantic Captioning Using Brain Alignments (BrainSCUBA): A technique unifying the embedding spaces of CLIP image and text embeddings with fMRI encoder weights to drive a vision-language model. This enables the generation of natural language descriptions of voxel-wise selectivity within the visual cortex.
3. Semantic Attribution and Image Localization (BrainSAIL): An approach employing vision foundation models and dense semantic features to localize activating objects within complex naturalistic images across higher-level visual areas.
These computational methods are complemented by human validation experiments using synthetically generated stimuli. My research paves the way towards a more comprehensive and ecologically valid understanding of visual processing, with implications for building more accurate models of the brain and contributing to novel applications in AI.
Bio
Andrew Luo is currently an Assistant Professor in the Institute of Data Science (IDS) & Psychology at the University of Hong Kong (HKU). He obtained his PhD jointly in Machine Learning and Neural Computation from Carnegie Mellon University (CMU), and a Bachelor's degree in Computer Science from the Massachusetts Institute of Technology (MIT). His current research focuses on understanding representations in the human visual cortex, and leveraging insights from human cognition to improve generative models. His research areas include computational neuroscience, 3D generative models, and scene representations. He has first author publications in NeurIPS (1 oral), ICLR, CVPR (1 oral), and ICCV.
