I am an Assistant Professor in the AI thrust at The Hong Kong University of Science and Technology (Guangzhou campus).
I lead the Precognition Lab.
I am also an Affiliate Assistant Professor at HKUST computer science & engineering department.
Prior to joining HKUST-GZ, I was a senior researcher at Tencent Youtu Lab working with Chunhua Shen. I completed my Ph.D. in 2021 from Carnegie Mellon University, advised by Alexander Hauptmann.
My Ph.D. and research work were mostly funded by IARPA, NIST and NSF grants.
I was a research intern at Google AI multiple times and collaborated with Lu Jiang, Liangliang Cao, Jia Li and Kevin Murphy.
I obtained my undergraduate degree from RUC, advised by Qin Jin.
I am looking for one PhD. student to work on Embodied AI. Send me an email if you are interested and have two or more first-authored published papers at top conferences.
My lab is at capacity now. I will look for PhD. students from MPhils of the university.
Please checkout our lab resources and this letter. 招生文: [知乎] [小红书]
For fellow assistant professors and Ph.D. students, I have gathered some awesome lists of resources that may be useful for your success. See here and feel free to contribute on Github. [被CVer盗了hhh]
If you want to meet, check out my public calendar first and propose a meeting via email.
Our research lab, the Precognition Lab (智能感知与预测实验室), is interested in building human-level Embodied AI systems that can effectively perceive, reason, and interact with the real-world for the good of humans.
Here is an up-to-date research roadmap.
Here are our on-going or finished research grants.
Our lab's computing resources include 32 RTX 3090/4090 GPUs and a cluster of 24 A6000 GPUs with a 100TB NAS. See this post.
And we have multiple mobile platforms with robot arms and dex hands:
- 11/2024 Presented "Towards General Service Embodied AI" at ARTS 2024. [自主机器人技术研讨会]
- 11/2024 My first PhD student, Xiaoyu Zhu, has successfully defended her thesis and will graduate from CMU. Congrats to Xiaoyu! [Learning Generalizable Visual Representations Towards Novel Viewpoints, Scenes and Vocabularies]
- 10/2024 Presented "Towards General Service Embodied AI" at Huawei and CCF-YOCSEF seminar.
- 09/2024 Presented "Towards General Service Embodied AI" at CCF/CSIG GAMES Seminar. [第九届计算机图形学与混合现实研讨会]
- 09/2024 One paper accepted at CoRL 2024.
- 09/2024 One paper accepted at NeurIPS 2024.
- 08/2024 One paper accepted at IROS 2024.
- 09/2024 被香港科技大学(广州)工会,评为笃志星(引领星):D [学校新闻]
- 09/2024 Presented "Towards General Service Embodied AI" at CAA's Seminar. [CAA中国自动化学会云讲座]
- 08/2024 Presented "Towards General Service Embodied AI" at CAAI's Embodied AI Seminar. [CAAI中国人工智能学会具身智能青年学者研讨会第五期] [Video Recording]
- 07/2024 Presented "Towards General Service Embodied AI" at the World AI Conference in Shanghai. [WAIC] [联汇科技]
- 07/2024 Two papers accepted at ECCV 2024.
- 05/2024 1 paper accepted at NAACL 2024. 1 paper accepted at ACL 2024. Main conference.
- 04/2024 My PhD student Dicong Qiu is reported by the university media. [HKUST(GZ)] [INFO Hub] [AI Thrust]
- 02/2024 Serve as a panelist at the VALSE Embodied AI webinar. [VALSE] [bilibili]
- 02/2024 Co-organizing the The 6th workshop on Precognition: Seeing through the Future @CVPR 2024. [Call For Papers] [知乎] [小红书]
- 12/2023 Keynote speech at the CEII2023 Workshop [Schedule]
- 10/2023 Co-organizing the Open-world Visual Perception Workshop (“开放世界下的视觉感知和增强”主题论坛) @PRCV 2023 [Schedule]
- 09/2023 Hosting HKUST AI Seminar series. Many thanks to the incoming speakers from around the world! [Course Website]
- 08/2023 HKUST-GZ PhD Summer Camp has started! Welcome! [Project 1] [Project 2]
- 07/2023 Attended a series of talks and events. No more traveling till fall! [WAIC @Shanghai] [RUC Seminar @Beijing]
- 06/2023 The Precognition Workshop was successfully held at CVPR! Thanks to all the co-organizers and program committee members! [Workshop Site] [CVPR Workshop Recording]
- 02/2023 One paper accepted by CVPR 2023. Congrats to Xiaoyu!
- 02/2023 I'm teaching AIAA 5032 Foundations of Artificial Intelligence and AIAA 5036 Autonomous AI this semester at HKUST (Guangzhou).
- 01/2023 I am co-organizing the The 5th workshop on Precognition: Seeing through the Future @CVPR 2023. [Call For Papers] [知乎]
- 10/2022 Presented first-ever lecture at HKUST (Guangzhou). [AI Seminar]
- 10/2022 Two papers accepted at NeurIPS 2022. [Multi-Action (Spotlight paper, 3.7% acceptance rate, 384/10411)] [Video Retrieval]
- 10/2022 Joined HKUST-GZ as a Tenure-Track Assistant Professor. Started an awesome list collection for TTAPs and PhD students.
- 09/2022 Invited to present at a young researcher forum by Prof. Xiaoou Tang and Shanghai AI Lab.
- 06/2022 Achieved second-place out of 150 teams on the public leaderboard of the Naturalist Driver Action Recognition Task - AI City Challenge @ CVPR 2022. [CVPRW Paper] [Presentation] [Code and Model]
- 10/2021 Published a research talk at TechBeat.net on Pedestrian Trajectory Prediction. [将门TechBeat] [B站]
- 08/2021 Received Doctoral Consortium Award at ICCV 2021, mentored by Prof. Alexandre Alahi.
- 08/2021 1 paper accepted by ICCV 2021.
- 08/2021 Our VERA system helps another major Washington Post news report. [link]
- 07/2021 Successfully defended my Ph.D. thesis: From Recognition to Prediction: Analysis of Human Action and Trajectory Prediction in Video. [link]
- 04/2021 Featured in a front-page news report (04/15) by Washington Post using crowding counting technologies. [video] [知乎]
- 01/2021 Invited presentation at ICPR'20 pattern forecasting workshop. [link]
- 09/2020 We won the Automated Streams Analysis for Public Safety Challenge with a $30k prize.
- 08/2020 Our paper has been accepted by WACV 2021 (one strong-accept) and reported by CMU news:
- 08/2020 Analyzed videos for journalist from the Washington Post on a major news.
- 07/2020 Awarded "AI Rising Star" at the World AI Conference.
- 07/2020 SimAug paper accepted by ECCV 2020.
- 06/2020 Multiverse (CVPR 2020) code and dataset are released! [blog] [知乎] [code]
- 12/2019 Received Baidu Scholarship (10 recipients globally). Press Coverage: , , , ,
-
09/2019 Our Shooter Localization System won Best Demo award at CBMI2019. [Project Site]
Press Coverage: , , , , , - 06/2019 Presented Future Prediction paper at CVPR 2019. It was reported by the media and it received 30k+ views in a week. [Tweets]
- 04/2019 Our CMU team's (INF & MUDSML) system achieved the best performance on the activity detection challenge (Cached) in surveillance videos hosted by NIST & IARPA. We have released our code and model for Object Detection & Tracking here.
- 12/2018 MemexQA paper accepted by TPAMI 2019.
- 06/2018 Presented MemexQA paper at CVPR 2018. [Spotlight Talk]
- 11/2016 Best performer in the NIST TRECVID 2016 Ad-hoc Video Search Challenge (no annotation track).
- ICCV Doctoral Consortium Award 2021
-
Rising Star (云帆奖-明日之星), World AI Conference 2020
- Baidu Scholarship (10 Ph.D. students worldwide) 2019
- Winner, Automated Streams Analysis for Public Safety Challenge - $30k prize 2020
- Best Demo Award at CBMI2019 2019
- Yahoo! Fellowship 2016 - 2018
- Winner, TRECVID ActEV Challenge 2019
- Winner, TRECVID Ad-hoc Video Search Challenge, no annotation track 2016
- CMU LTI Student Research Symposium Best Paper Honorable Mentions 2018
- Google Cloud COVID-19 Research Grant - $6200 2020
-
For more up-to-date media coverage, please visit my lab website.
- Washington Post. How Shireen Abu Akleh was killed (provided gunshot and shooter analysis), June 2022. [Link]
- Washington Post. Anatomy of a crackdown (provided gunshot and shooter analysis), August 25, 2021. [Link]
- Washington Post. 17 requests for backup in 78 minutes (provided crowd counting analysis), April 15, 2021. [Link]
- Carnegie Mellon University News. Amateur Drone Videos Could Aid in Natural Disaster Damage Assessment, August 28, 2020.
- AZO Robotics. New AI System Helps Detect Damage Caused to Buildings by Hurricanes, August 31, 2020.
- Washington Post. Lewd cheerleader videos, sexist rules: Ex-employees decry Washington’s NFL team workplace (featured in the video analytics), August 26, 2020. [Link]
- CBS. Researchers At Carnegie Mellon University Develop Video System To Locate Mass Shooters Using Smartphones, November 20, 2019.
- post-gazette. CMU develops video system that can locate mass shooter, November 20, 2019.
- GIZMODO. Smartphone Videos Can Now Be Analyzed and Used to Pinpoint the Location of a Shooter, November 21, 2019.
- DailyMail. Active shooters can be located within minutes by new software that analyzes smartphone video from the scene and can even identify the type of gun, November 20, 2019.
- Techspot. Researchers develop system that can pinpoint a shooter's location using smartphone videos, November 21, 2019.
- New York Times. Who Killed the Kiev Protesters? A 3-D Model Holds the Clues (featured in the video analytics), May 30, 2018.
- 读芯术. 卡内基梅隆大学梁俊卫:视频中行人的多种未来轨迹预测, August, 2020.
- Baidu. 乘风破浪的AI技术青年——首届WAIC云帆奖名单公布, July 11, 2020.
- China.com.cn. 人大高瓴人工智能学院“高屋建瓴-青年说”首期开讲, Jan 6, 2020.
- Baidu. AI界的中国力量!百度奖学金助力中国AI人才绽放光芒!, Jan 5, 2020.
- 量子位. 李飞飞团队造出”窥视未来”新AI:去哪干啥一起猜, 准确率压倒老前辈, received 30k+ views in a week, Feb 13, 2019.
- 机器之心. 遇见未来!李飞飞等提出端到端系统Next预测未来路径与活动, Feb 14, 2019.
-
Aminer.cn, AI 2000 ranking (2019 - 2022).