Staff profile

🏠Homepage 🎓Google Scholar 🎞️YouTube 🔗LinkedIn 𝕏 Twitter 💻Github ✉️Email

Research Interests: Responsible AI, Computer Vision, Computer Graphics, AI in Healthcare

Prof. Hubert P. H. Shum is a Professor of Visual Computing and the Director of Research of the Department of Computer Science at Durham University, specialising in modelling spatio-temporal information with responsible AI. He has authored over 200 research publications in the fields of Computer Vision, Computer Graphics and AI in Healthcare, underpinned by Responsible AI designs and algorithms. His work has been featured in leading venues such as CVPR, ICCV, ECCV, SIGGRAPH, MICCAI, IEEE TVCG and IEEE TIP.

His research is fundamental to many disciplines, facilitating him to engage in multidisciplinary research related to healthcare, space technology, art, autonomous vehicles and robotics. He is a Co-Founder and the Co-Director of Durham University Space Research Centre, a Fellow of the Wolfson Research Institute for Health and Wellbeing, and a Steering Group Member of the Centre for Visual Arts and Culture. He is an Advisory Board member of TORUS, a £6 mil EPSRC project supporting Parkinson's disease patients with sensing technologies.

To develop his team, he has led research projects as the Principal Investigator awarded by EPSRC, the Ministry of Defence, the Royal Society and Innovate UK. He has also been a Co-Investigator in NortHFutures, a £4.17 mil EPSRC project to establish a new digital health hub in North East England. These have facilitated him to have supervised more than 30 PhDs, recruited multiple Post-doctoral Researchers, and collaborated with international researchers from the UK, France, China, Japan and India.

To engage the academic and industrial networks, he has chaired international conferences such as Pacific Graphics, BMVC, ACM SIGGRAPH/Eurographics SCA and ACM SIGGRAPH MIG. He has been interviewed by the UK Catapult Network for his counter-drone research. Contributing to the research community, he has served as an Associate Editor for the Computer Graphics Forum, a Guest Editor for the International Journal of Computer Vision, and a Program Committee member in over 20 conferences such as Eurographics and Pacific Graphics. He also has also served in the EPSRC Peer Review College and has frequently reviewed grant proposals from EPSRC, Mitacs, ANR, NSERC, FCT.

Joining Us

Follow the application instructions to join us either as a PhD student, a visiting/internship student, or an academic visitor. My team has a supportive culture with members developing high-quality research while enjoying their vibrant lives in the UK.

PhD directions include:

• Computer Vision with Responsible Artificial Intelligence for Human-Related Data
• Deep Learning Based Machine Vision for Space-Informed Applications
• Artificial Intelligence for Virtual Characters in Computer Graphics and Computer Animation

Research Grants

• EPSRC Impact Acceleration Accounts (Ref: 2960307), "A Pilot Study To Deliver Healthcare Innovation For Diabetic Eye Disease In The North East", Co-Investigator,
• Durham University Strategy Research Fund (Ref: 1755302), "Durham University Space Research Centre", Co-Founder and Co-Investigator, Department of Computer Science Leader, £2.76 million
• EPSRC Impact Acceleration Accounts (Ref: 2312537), "Advancing 360 Degree Real-time Video Surveillance with Automated Anomaly Detection", Co-Investigator
• EPSRC Digital Health Hub Pilot Scheme (Ref: EP/X031012/1), "Northern Health Futures Hub (NortHFutures)", Co-investigator, £4.17 million
• Knowledge Transfer Partnership (Ref: 10050391), "Skin Lesion Classification with Real-World Data", Principal Investigator, £143,393
• Defence and Security Accelerator (Ref: DSTL0000007030, ACC6031106), "Autonomous Persistent Wide Area Surveillance of Human and Vehicle Activity", Co-investigator, £93,978
• Security Technology Research Innovation Grants Programme (Ref: 007CD), "Tracking Drones Across Different Platforms with Machine Vision", Principal Investigator
• Defence and Security Accelerator (Ref: DSTLX-1000140725), "D-FOCUS: Drone-FOrmation Control for countering future Unmanned aerial Systems", Principal Investigator, £124,901
• GX Project Grants Scheme, "Wound Monitoring with Depth Cameras on Portable Systems", Principal Investigator
• Royal Society International Exchanges (Ref: IES\R2\181024), "Modelling Human Motion for Synthesis and Recognition with Deep Learning on Surface Features", Principal Investigator
• Royal Society International Exchanges (Ref: IE160609), "An Affective Smart Environment for Personalized Learning and Teaching", Principal Investigator
• Erasmus Mundus Action 2 Programme (Ref: 2014-0861/001-001), "Sustainable Green Economies through Learning, Innovation, Networking and Knowledge Exchange (gLink)", Northumbria University Funding Management Officer, €3.03 million
• EPSRC (Ref: EP/M002632/1), "Interaction-based Human Motion Analysis", Principal Investigator, £123,819

Recent Research

• Adaptive Graph Learning from Spatial Information for Surgical Workflow Anticipation (IEEE Transactions on Medical Robotics and Bionics, 2025)

• Multi-Task Spatial-Temporal Graph Auto-Encoder for Hand Motion Denoising (IEEE Transactions on Visualization and Computer Graphics, 2024)

• HINT: High-quality INpainting Transformer with Mask-Aware Encoding and Enhanced Attention (IEEE Transactions on Multimedia, 2024)

• Geometric Features Enhanced Human-Object Interaction Detection (IEEE Transactions on Instrumentation and Measurement, 2024)

• RAPiD-Seg: Range-Aware Pointwise Distance Distribution Networks for 3D LiDAR Segmentation (Proceedings of the 2024 European Conference on Computer Vision, Oral Paper)

• MAGR: Manifold-Aligned Graph Regularization for Continual Action Quality Assessment (Proceedings of the 2024 European Conference on Computer Vision, Oral Paper)

• Depth-Aware Endoscopic Video Inpainting (Proceedings of the 2024 International Conference on Medical Image Computing and Computer Assisted Intervention)

• A Video-Based Augmented Reality System for Human-in-the-Loop Muscle Strength Assessment of Juvenile Dermatomyositis (IEEE Transactions on Visualization and Computer Graphics, 2023)

• Hierarchical Graph Convolutional Networks for Action Quality Assessment (IEEE Transactions on Circuits and Systems for Video Technology)

• Interaction-Aware Decision-Making for Automated Vehicles using Social Value Orientation (IEEE Transactions on Intelligent Vehicles, 2023)

• Less is More: Reducing Task and Model Complexity for 3D Point Cloud Semantic Segmentation (Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition)

• Hard No-Box Adversarial Attack on Skeleton-Based Human Action Recognition with Skeleton-Motion-Informed Gradient (Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision)

• Unaligned 2D to 3D Translation with Conditional Vector-Quantized Code Diffusion using Transformers (Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision)

• A Pose-Based Feature Fusion and Classification Framework for the Early Prediction of Cerebral Palsy in Infants (IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2022)

• Geometric Features Informed Multi-Person Human-Object Interaction Recognition in Videos (Proceedings of the 2022 European Conference on Computer Vision)

• Pose-Based Tremor Classification for Parkinson’s Disease Diagnosis from Video (Proceedings of the 2022 International Conference on Medical Image Computing and Computer Assisted Intervention)

• 360 Depth Estimation in the Wild - The Depth360 Dataset and the SegFuse Network (Proceedings of the 2022 IEEE Conference on Virtual Reality and 3D User Interfaces)

Professional Activities

Conference Services

• Conference Chair of Pacific Conference on Computer Graphics and Applications 2022
• Conference Chair of ACM SIGGRAPH/Eurographics Symposium on Computer Animation 2022
• Conference Chair of the ACM SIGGRAPH Conference on Motion, Interaction and Games 2019
• Program Chair of the British Machine Vision Conference 2018
• Program Chair of the International Conference on Software, Knowledge, Information Management and Applications 2018
• Program Chair of the ACM SIGGRAPH Conference on Motion in Games 2016
• Co-Organiser of IEEE/CVF Winter Conference on Applications of Computer Vision Workshop on Human-Autonomous Vehicle Interaction 2025
• Demo and Poster Chair of the ACM Symposium on Virtual Reality Software and Technology 2018
• Area Chair of the British Machine Vision Conference 2024, 2023, 2022, 2021 and 2020

Journal Editorships

• Computer Graphics Forum (CGF), Associate Editor, 2019 - 2023
• International Journal of Computer Vision (IJCV), Guest Editor, Special Issue on Machine Vision and Deep Learning, 2020

Awards

• The Sullivan Doctoral Thesis Prize runner-up for the co-supervised PhD Li Li's PhD thesis.
• Best Paper Award in the IEEE/CVF Conference on Computer Vision and Pattern Recognition DCAMI Workshop, 2024
• Exceptional Achievement for Excellence in PhD Supervision by Durham University in 2023
• Exceptional Achievement for Excellence in Teaching by Durham University in 2022
• Best Paper Award in the IEEE International Conference on Human-Machine Systems 2021
• Best Student Paper Award in the International Conference on Computer Graphics Theory and Applications (GRAPP) 2020
• Best Poster Award in the ACM SIGGRAPH Conference on Motion, Interaction and Games (MIG) 2019
• Best Paper Award in the International Conference on Advances in Computer Entertainment Technology (ACE) 2017
• Best Poster Award in the ACM SIGGGRAPH Conference on Motion in Games (MIG) 2016

Media Coverage

• Interviewed by Durham's 100 Faces of Science on AI for space research, 2024.
• Interviewed by the UK Catapult Network on how AI and computer vision enhance security, 2023

• Responsible AI
• Computer Vision
• Computer Graphics
• AI in Healthcare

• 2024: Director of Research, Department of Computer Science:
• 2024: Co-Founder and Co-Director of Space Research Centre:

Conference Paper

• Xiatian Zhang, F., Chen, S., Xie, X., & Shum, H. P. (2024, October). Depth-Aware Endoscopic Video Inpainting. Presented at 27th International Conference on Medical Image Computing and Computer Assisted Intervention, Marrakesh, Morocco
• Zhou, K., Wang, L., Zhang, X., Shum, H. P. H., Li, F. W. B., Li, J., & Liang, X. (2024, September). MAGR: Manifold-Aligned Graph Regularization for Continual Action Quality Assessment. Presented at ECCV 2024: The 18th European Conference on Computer Vision, Milan, Italy
• Li, L., Shum, H. P. H., & Breckon, T. P. (2024, September). RAPiD-Seg: Range-Aware Pointwise Distance Distribution Networks for 3D LiDAR Segmentation. Presented at ECCV 2024: European Conference on Computer Vision, Milan, Italy
• Feng, Q., Shum, H. P., & Morishima, S. (2023, October). Enhancing Perception and Immersion in Pre-Captured Environments through Learning-Based Eye Height Adaptation. Presented at ISMAR 23: International Symposium on Mixed and Augmented Reality, Sydney, Australia
• Lu, Z., Wang, H., Chang, Z., Yang, G., & Shum, H. P. (2023, October). Hard No-Box Adversarial Attack on Skeleton-Based Human Action Recognition with Skeleton-Motion-Informed Gradient. Presented at ICCV 2023: 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France
• Li, L., Shum, H. P., & Breckon, T. P. (2023, June). Less is More: Reducing Task and Model Complexity for 3D Point Cloud Semantic Segmentation. Presented at 2023 IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR), Vancouver, BC
• Corona-Figueroa, A., Bond-Taylor, S., Bhowmik, N., Gaus, Y. F. A., Breckon, T. P., Shum, H. P., & Willcocks, C. G. (2023, October). Unaligned 2D to 3D Translation with Conditional Vector-Quantized Code Diffusion using Transformers. Presented at ICCV23: 2023 IEEE/CVF International Conference on Computer Vision, Paris, France
• Zhang, X., Zhang, H., & Shum, H. P. (2022, September). Pose-based Tremor Classification for Parkinson’s Disease Diagnosis from Video. Presented at MICCAI '22: The 25th International Conference on Medical Image Computing and Computer Assisted Intervention, Singapore
• Qiao, T., Men, Q., Li, F. W., Kubotani, Y., Morishima, S., & Shum, H. P. (2022, October). Geometric Features Informed Multi-person Human-object Interaction Recognition in Videos. Presented at Computer Vision - ECCV 2022, Tel Aviv, Israel
• Feng, Q., Shum, H. P., & Morishima, S. (2022, March). 360 Depth Estimation in the Wild - The Depth360 Dataset and the SegFuse Network. Presented at IEEE Conference on Virtual Reality and 3D User Interfaces, Christchurch, New Zealand
• Leng, Z., Chen, J., Shum, H. P., Li, F. W., & Liang, X. (2021, March). Stable Hand Pose Estimation under Tremor via Graph Neural Network. Presented at 2021 IEEE Virtual Reality and 3D User Interfaces (VR), Lisboa

Journal Article

• Zhang, F. X., Deng, J., Lieck, R., & Shum, H. P. (online). Adaptive Graph Learning from Spatial Information for Surgical Workflow Anticipation. IEEE Transactions on Medical Robotics and Bionics, https://doi.org/10.1109/TMRB.2024.3517137
• Li, R., Qiao, T., Katsigiannis, S., Zhu, Z., & Shum, H. (in press). Unified Spatial-Temporal Edge-Enhanced Graph Networks for Pedestrian Trajectory Prediction. IEEE Transactions on Circuits and Systems for Video Technology,
• Zhou, K., Ma, Y., Shum, H. P., & Liang, X. (online). Hierarchical Graph Convolutional Networks for Action Quality Assessment. IEEE Transactions on Circuits and Systems for Video Technology, 33(12), 7749 - 7763. https://doi.org/10.1109/TCSVT.2023.3281413
• Zhou, K., Shum, H. P., Li, F. W., & Liang, X. (2024). Multi-Task Spatial-Temporal Graph Auto-Encoder for Hand Motion Denoising. IEEE Transactions on Visualization and Computer Graphics, 30(10), 6754-6769. https://doi.org/10.1109/TVCG.2023.3337868
• Chen, S., Atapour-Abarghouei, A., & Shum, H. P. H. (2024). HINT: High-quality INpainting Transformer with Mask-Aware Encoding and Enhanced Attention. IEEE Transactions on Multimedia, 26, 7649-7660. https://doi.org/10.1109/TMM.2024.3369897
• Zhu, M., Ho, E. S. L., Chen, S., Yang, L., & Shum, H. P. H. (2024). Geometric Features Enhanced Human-Object Interaction Detection. IEEE Transactions on Instrumentation and Measurement, 73, Article 5026014. https://doi.org/10.1109/TIM.2024.3427800
• Zhou, K., Cai, R., Ma, Y., Tan, Q., Wang, X., Li, J., Shum, H. P., Li, F. W., Jin, S., & Liang, X. (2023). A Video-Based Augmented Reality System for Human-in-the-Loop Muscle Strength Assessment of Juvenile Dermatomyositis. IEEE Transactions on Visualization and Computer Graphics, 29(5), 2456-2466. https://doi.org/10.1109/tvcg.2023.3247092
• Crosato, L., Shum, H. P., Ho, E. S., & Wei, C. (2023). Interaction-aware Decision-making for Automated Vehicles using Social Value Orientation. IEEE Transactions on Intelligent Vehicles, 8(2), 1339-1349. https://doi.org/10.1109/tiv.2022.3189836
• McCay, K. D., Hu, P., Shum, H. P., Woo, W. L., Marcroft, C., Embleton, N. D., Munteanu, A., & Ho, E. S. (2022). A Pose-based Feature Fusion and Classification Framework for the Early Prediction of Cerebral Palsy in Infants. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 30, 8-19. https://doi.org/10.1109/tnsre.2021.3138185
• Men, Q., Ho, E. S., Shum, H. P., & Leung, H. (2021). A Quadruple Diffusion Convolutional Recurrent Network for Human Motion Prediction. IEEE Transactions on Circuits and Systems for Video Technology, 31(9), 3417-3432. https://doi.org/10.1109/tcsvt.2020.3038145
• Wang, H., Ho, E. S., Shum, H. P., & Zhu, Z. (2021). Spatio-temporal Manifold Learning for Human Motions via Long-horizon Modeling. IEEE Transactions on Visualization and Computer Graphics, 27(1), 216 - 227. https://doi.org/10.1109/tvcg.2019.2936810
• Hu, S., Shum, H., Aslam, N., Li, F. W., & Liang, X. (2020). A Unified Deep Metric Representation for Mesh Saliency Detection and Non-rigid Shape Matching. IEEE Transactions on Multimedia, 22(9), 2278-2292. https://doi.org/10.1109/tmm.2019.2952983
• Shen, Y., Yang, L., Ho, E. S., & Shum, H. P. (2020). Interaction-Based Human Activity Comparison. IEEE Transactions on Visualization and Computer Graphics, 26(8), 2620-2633. https://doi.org/10.1109/tvcg.2019.2893247
• Kar, A., Pramanik, S., Chakraborty, A., Bhattacharjee, D., Ho, E. S., & Shum, H. P. (2020). LMZMPM: Local Modified Zernike Moment Per-unit Mass for Robust Human Face Recognition. IEEE Transactions on Information Forensics and Security, 16, 495-509. https://doi.org/10.1109/tifs.2020.3015552
• Rueangsirarak, W., Zhang, J., Aslam, N., Ho, E. S., & Shum, H. P. (2018). Automatic Musculoskeletal and Neurological Disorder Diagnosis With Relative Joint Displacement From Human Gait. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 26(12), 2387-2396. https://doi.org/10.1109/tnsre.2018.2880871
• Zhang, J., Shum, H. P., Han, J., & Shao, L. (2018). Action Recognition From Arbitrary Views Using Transferable Dictionary Learning. IEEE Transactions on Image Processing, 27(10), 4709-4723. https://doi.org/10.1109/tip.2018.2836323
• Zhang, L., Shum, H. P., & Shao, L. (2017). Manifold Regularized Experimental Design for Active Learning. IEEE Transactions on Image Processing, 26(2), 969-981. https://doi.org/10.1109/tip.2016.2635440
• Liu, Z., Zhou, L., Leung, H., & Shum, H. P. (2016). Kinect Posture Reconstruction Based on a Local Mixture of Gaussian Process Models. IEEE Transactions on Visualization and Computer Graphics, 22(11), 2437-2450. https://doi.org/10.1109/tvcg.2015.2510000
• Zhang, L., Shum, H., & Shao, L. (2016). Discriminative Semantic Subspace Analysis for Relevance Feedback. IEEE Transactions on Image Processing, 25(3), 1275-1287. https://doi.org/10.1109/tip.2016.2516947
• Henry, J., Shum, H. P., & Komura, T. (2014). Interactive Formation Control in Complex Environments. IEEE Transactions on Visualization and Computer Graphics, 20(2), 211-222. https://doi.org/10.1109/tvcg.2013.116
• Shum, H. P., Ho, E. S., Jiang, Y., & Takagi, S. (2013). Real-Time Posture Reconstruction for Microsoft Kinect. IEEE Transactions on Cybernetics, 43(5), 1357-1369. https://doi.org/10.1109/tcyb.2013.2275945
• Shum, H. P., Komura, T., & Yamazaki, S. (2012). Simulating Multiple Character Interactions with Collaborative and Adversarial Goals. IEEE Transactions on Visualization and Computer Graphics, 18(5), 741-752. https://doi.org/10.1109/tvcg.2010.257
• Shum, H. P., Komura, T., Shiraishi, M., & Yamazaki, S. (2008). Interaction patches for multi-character animation. ACM Transactions on Graphics, 27(5), Article 114. https://doi.org/10.1145/1409060.1409067