Yisheng He is a research scientist at Tongyi Lab, Alibaba (Alibaba Star Project). Before that, he obtained his Ph.D. at HKUST.
We are hiring research interns. To apply, please email your CV to ethanheysh@gmail.com.
I'm interested in Embodied AI, Multimodal Generation, and 3D Vision.
* denotes equal contribution; ^ denotes intern student; † denotes project leader; ✉ denotes corresponding author.
Technical Report, 2026
λ₀ learns whole-body humanoid loco-manipulation through egocentric pretraining, mid-training on the 500-hour HumanVerse dataset with human-to-humanoid alignment, and robot post-training. Scaling whole-body human data improves downstream task performance.
Technical Report, 2026
EgoAlign adapts egocentric human demonstrations via controller-aware motion alignment and causal robot-state reconstruction, enabling human-only task training of VLA models and zero-shot deployment for long-range humanoid loco-manipulation, thereby serving as the cornerstone of λ₀.
ACM TOG, 2026
LAM-Talker creates an animatable 3D Gaussian head from a single image, combining audio-driven animation, real-time cross-platform rendering, and a speech dialogue system. This enables interactive talking avatars on devices including mobile phones.
NeurIPS, 2026
FA-LAM is a focus-aware large avatar model for one-shot animatable 3D Gaussian head and streaming 4D full-head reconstruction, with symmetric semantic attention regularization, a dual-phase training pipeline, and an autoregressive design with visibility-gated fusion.
NeurIPS, 2026
PASPA compresses motion history into a persistent state and models each new segment with bidirectional attention within a blockwise flow-matching framework. This preserves motion style and temporal coherence over long speech sequences with fixed-size history memory.
IEEE TVCG, 2026
PanoLAM is a large avatar model for Gaussian full-head reconstruction from a single unposed image. It uses a coarse-to-fine, dual-branch framework to reconstruct a Gaussian full head in under a second.
ECCV, 2026
Forge4D is the first feed-forward model for 4D human Gaussian reconstruction in real-world metric scale, and enables novel-view and novel-time synthesis from uncalibrated sparse-view videos in an efficient streaming manner.
CVPR, 2026
MeshLAM is a feed-forward framework for one-shot animatable mesh avatar reconstruction from a single image, using a dual shape‑texture architecture with iterative GRU decoding and reprojection‑based texture guidance.
IEEE TIP, 2026
We build a bidirectional control flow between the style and the content for stylized motion generation and enable multimodal style control including text, image, and style motions.
ICLR, 2026
LDCM is a transformer-based framework for metric-depth completion from sparse observations, with Poisson depth initialization and a point-map head to regress per‑pixel 3D coordinates without camera intrinsics.
Preprint, 2025
ViSA integrates 3D reconstruction priors with a real-time autoregressive video diffusion model to generate photorealistic, temporally coherent upper-body avatars from a single image for gaming and VR.
ACM MM, 2025
CoProSketch enables fine-grained control and detailed sketch generation with diffusion models.
SIGGRAPH, 2025
LAM creates animatable Gaussian heads with one-shot images in a single forward pass, which can be reenacted and rendered on various platforms (including mobile phones) in real time.
ICLR, 2025
LaMP is a language-motion pretraining model that advances text-to-motion generation, motion-text retrieval, and motion captioning through aligned language-motion representation learning.
NeurIPS, 2024 (Oral Presentation)
We introduce a hybrid framework that leverages 3D Gaussian representation to advance physical property identification.
NeurIPS, 2024
We introduce a 2D joint VQ-VAE to quantize each joint instead of all joints into tokens. A spatial-temporal modeling framework with temporal-spatial 2D masking and 2D attention is also proposed for motion generation.
ECCV, 2024
We enable high-fidelity, transferable neural field editing with controllable edit intensity.
arXiv, 2024
Our method synthesizes consistent 3D content with fine-grained sketch control.
IEEE Robotics and Automation Letters (RA-L), 2024
We introduce a new problem: open-vocabulary 9D object pose and size estimation, a new dataset: OO3D-9D, and a new framework based on a vision foundation model to tackle this problem.
arXiv, 2022
A self-supervised framework for category-level object pose and size estimation via differentiable shape deformation, registration, and rendering.
CVPR, 2022
A new open-set few-shot 6D object pose estimation problem: estimating the 6D pose of an unknown object by a few support views without CAD models and extra training. A large-scale synthetic dataset for pre-training and benchmarks for future research.
CVPR, 2021 (Oral Presentation)
paper code video (youtube) video (bilibili)
A generic full flow bidirectional fusion framework for RGBD representation learning, applied to joint instance semantic segmentation and 3D keypoint-based 6D pose estimation.
arXiv, 2021
A brand new dataset to promote the study of instance segmentation for objects with irregular shapes and an affinity-based algorithm to tackle it.
CVPR, 2020
paper code video (youtube) video (bilibili)
The first deep learning 3D keypoint-based 6D pose estimation algorithm and an overall framework for joint instance semantic segmentation and 3D keypoint detection.
Tongyi Lab, Alibaba Group
Alibaba Star Program, 1%
Megvii Technology (Face++)
Supervisor: Dr. Jian Sun, Chief Scientist, Megvii Research
Mentor: Haoqiang Fan, Megvii Research
Collaborator: Dr. Haibin Huang, Megvii Research
Mentors: Haoqiang Fan and Dr. Yuzhi Wang, Megvii Research
Microsoft
Mentors: Raymond Xue and Hao Lin, Microsoft
