Yisheng He (何益升)

Yisheng He is a forth year Ph.D. student at the Hong Kong University of Science and Technology (HKUST), advised by Prof. Qifeng Chen, Prof. Long Quan, and Dr. Jian Sun. He also collaborates with Dr. Haibin Huang and Haoqiang Fan.

Email  /  Google Scholar  /  GitHub  /  WeChat

profile photo

I'm interested in 3D computer vision, RGBD representation learning (sensor fusion), robotics, few-shot learning and self-supervised learning. Downstream applications includes autonomous driving, AR/VR, robotic manipulation, etc.

clean-usnob Full Flow Bidirectional Fusion For 3D Keypoint-Based 6D Pose Estimation
Yisheng He, Haibin Huang, Haoqiang Fan, Qifeng Chen, Jian Sun

We extend FFB6D to deal with multi instances of the same class and improve keypoint detection on symmetric objects.

clean-usnob Towards Self-Supervised Category-Level Object Pose and Size Estimation
Yisheng He, Haoqiang Fan, Haibin Huang, Qifeng Chen, Jian Sun
In submision, 2022
project page / arXiv

A self-supervised framework for category-level object pose and size estimation via differentiable shape deformation, registration, and rendering.

clean-usnob FS6D: Few-Shot 6D Pose Estimation of Novel Objects
Yisheng He, Yao Wang, Haoqiang Fan, Jian Sun, Qifeng Chen
CVPR, 2022
project page / arXiv / data / code GitHub stars

A new open-set few-shot 6D object pose estimation problem: estimating the 6D pose of an unknown object by a few support views without CAD models and extra training. A large-scale synthesis dataset for pre-training and benchmarks for future research.

clean-usnob FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation
Yisheng He, Haibin Huang, Haoqiang Fan, Qifeng Chen, Jian Sun
CVPR, 2021 (Oral Presentation)
project page / arXiv / code GitHub stars / video (youtube) / video (bilibili)

A generic full flow bidirectional fusion framework for RGBD representation learning, applied to joint instance semantic segmentation and 3D keypoint-based 6D pose estimation.

clean-usnob iShape: A First Step Towards Irregular Shape Instance Segmentation
Lei Yang, Ziwei Yan, Yisheng He, Wei Sun, Zhenhang Huang, Haibin Huang, Haoqiang Fan
arXiv, 2021
project page / arXiv / code / dataset

A brand new dataset to promote the study of instance segmentation for objects with irregular shapes and an affinity-based algorithm to tackle it.

clean-usnob PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimation
Yisheng He, Wei Sun, Haibin Huang, Jianran Liu, Haoqiang Fan, Jian Sun
CVPR, 2020
project page / arXiv / code GitHub stars / video (youtube) / video (bilibili)

The first deep learning 3D keypoint-based 6D pose estimation algorithm and an overall framework for joint instance semantic segmantation and 3D keypoint detection.


I've also worked on technology that transfers to industrial products while at Megvii Research and Microsoft.

clean-usnob 3D Face Recognition | Liveness Detection

We developed the world's first 3D face recognition algorithm for Android smartphones. It's shipped with OPPO Find X announced on June 2018. I developed liveness detection algorithms based on depth and infrared (IR) images in the project.

The project team won the annual Meg-Team award, 2018.

clean-usnob Microsoft 365 Micro Asistant (Dragon-Gate)

We developed a set of office suites based on Microsoft Office 365 and WeChat. The product was announced on November, 2017.

clean-usnob Megvii (Face++), Senior Research Intern, Jan.2019-

Supervisor: Dr. Jian Sun; Collaborators: Dr. Haibin Huang, Haoqiang Fan

Megvii (Face++), Research Intern, Dec.2017-May.2018

Mentors: Dr. Yuzhi Wang, Haoqiang Fan

clean-usnob Microsoft, SDE Intern, Jun.2017-Oct.2017

Mentors: Raymond Xue, Hao Lin

Academic Challenge
clean-usnob Rank 2nd in OCRTOC: Open Cloud Robot Table Organization Challenge , 2020

  • CN108921070A (Issued 2018.11), "Image processing, model training methods and corresponding devices".
  • CN109191802A (Issued 2019.01), "Method, apparatus, system and storage medium for eye protection".
  • CN112614134A (In process), "Image segmentation method, device, electronic device and storage medium".
  • Services

  • Program Committee/Reviewers: CVPR, ECCV, NeurIPS, IROS, Neurocomputing
  • Teaching Assistant @ HKUST: COMP 4201 (Spring 2019), COMP 1029 (Fall 2020), COMP 4201 (Spring 2021)

  • Last updated: March, 2022.

    Thanks Dr. Jon Barron for sharing the template code.