Sameh Khamis
Co-Founder & CTO, Halfpixel
I’m a computer vision, graphics and machine learning researcher building real-time 3D systems. At Halfpixel, I’m building a real-world spatial intelligence platform that reconstructs, understands and predicts the physical world from pixels and sensors.
Previously, I co-founded Rembrand and worked at NVIDIA, Google and Microsoft Research. I earned my PhD in Computer Science at the University of Maryland. I’m based in the Bay Area.
Experience
Halfpixel · Co-Founder & CTO
present · Bay Area, CA · halfpixel.ai
- Building a real-world spatial intelligence platform: reconstructing, understanding and predicting the physical world from pixels and every sensor that sees it.
Rembrand · Co-Founder, Director of R&D
2023 – 2026 · Bay Area, CA · grew to 30+ people, $32M raised
- Led a 13-person team of vision & graphics engineers and VFX artists shipping generative AI for video: photoreal product placement in existing footage.
- Deployed SoTA systems for single-image volumetric lighting estimation, multi-shot camera pose tracking, and video object segmentation.
- Wrote a WebGL real-time progressive ray tracer for physically-based object insertion — inter-reflections and shadows included.
- Post-trained reference-guided video diffusion transformers on ~100 TB of raw video, filtered and processed through our own data pipelines.
NVIDIA · Principal Research Scientist
2020 – 2023 · Santa Clara, CA
- Digital twins & digital humans: research for photoreal avatars and 3D content creation in Omniverse.
- 3D-aware GANs (EG3D), one-shot Live 3D Portraits, relightable articulated avatars (RANA), virtual light stages.
- 3D style transfer (3DStyleNet), hybrid differentiable rendering (DIB-R++), neural kernel fields.
- I am AI — SIGGRAPH 2021 Real-Time Live, Best-in-Show.
Google AR/VR · Senior Research Scientist
2017 – 2020 · San Francisco, CA
- Tech lead for machine learning in ARCore — enabling AR on smartphones. Video
- Built the infrastructure for cross-dataset training, model distillation and compression.
- Efficient depth estimation from passive & active stereo, from motion, and from a single image; human pose estimation, neural rendering, image matting.
perceptiveIO · Senior Scientist, Founding Team
2016 – 2017 · San Francisco, CA · acquired by Google
- Built a state-of-the-art 360° performance capture system for real-time reconstruction of arbitrary non-rigid scenes.
Microsoft Research · Researcher, Interactive 3D Technologies
2015 – 2016 · Redmond, WA
- Holoportation — live on stage at TED 2016; covered by Wired, Engadget, ZDNet, Gizmodo, VentureBeat and Mashable. Video
- Hand tracking and hand-shape modeling from depth cameras.
Publications
Full list on Google Scholar and DBLP
2026
- iLRM: An Iterative Large 3D Reconstruction Model. CVPR 2026 arXiv · Project · Code
Gyeongjin Kang, Seungtae Nam, Seungkwon Yang, Xiangyu Sun, Sameh Khamis, Abdelrahman Mohamed, Eunbyung Park
2023
- Live 3D Portrait: Real-Time Radiance Fields for Single-Image Portrait View Synthesis. SIGGRAPH 2023 arXiv · Project
Alex Trevithick, Matthew Chan, Michael Stengel, Eric R. Chan, Chao Liu, Zhiding Yu, Sameh Khamis, Manmohan Chandraker, Ravi Ramamoorthi, Koki Nagano - RANA: Relightable Articulated Neural Avatars. ICCV 2023 arXiv · Project
Umar Iqbal, Akin Caliskan, Koki Nagano, Sameh Khamis, Pavlo Molchanov, Jan Kautz - Single-Shot Implicit Morphable Faces with Consistent Texture Parameterization. SIGGRAPH 2023 arXiv · Project
Connor Z. Lin, Koki Nagano, Jan Kautz, Eric R. Chan, Umar Iqbal, Leonidas Guibas, Gordon Wetzstein, Sameh Khamis - Learning Human Dynamics in Autonomous Driving Scenarios. ICCV 2023 PDF
Jingbo Wang, Ye Yuan, Zhengyi Luo, Kevin Xie, Dahua Lin, Umar Iqbal, Sanja Fidler, Sameh Khamis
2022
- EG3D: Efficient Geometry-aware 3D Generative Adversarial Networks. CVPR 2022 Oral arXiv · Project · Code
Eric R. Chan, Connor Z. Lin, Matthew A. Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas Guibas, Jonathan Tremblay, Sameh Khamis, Tero Karras, Gordon Wetzstein - Learning to Relight Portrait Images via a Virtual Light Stage and Synthetic-to-Real Adaptation. SIGGRAPH Asia 2022 arXiv · Project
Yu-Ying Yeh, Koki Nagano, Sameh Khamis, Jan Kautz, Ming-Yu Liu, Ting-Chun Wang - Neural Fields as Learnable Kernels for 3D Reconstruction. CVPR 2022 arXiv · Project
Francis Williams, Zan Gojcic, Sameh Khamis, Denis Zorin, Joan Bruna, Sanja Fidler, Or Litany - Frame Averaging for Equivariant Shape Space Learning. CVPR 2022 arXiv · Project
Matan Atzmon, Koki Nagano, Sanja Fidler, Sameh Khamis, Yaron Lipman
2021
- DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderer. NeurIPS 2021 arXiv · Project
Wenzheng Chen, Joey Litalien, Jun Gao, Zian Wang, Clement Fuji Tsang, Sameh Khamis, Or Litany, Sanja Fidler - 3DStyleNet: Creating 3D Shapes with Geometric and Texture Style Variations. ICCV 2021 Oral arXiv · Project
Kangxue Yin, Jun Gao, Maria Shugrina, Sameh Khamis, Sanja Fidler - I am AI: AI-driven Digital Avatar Made Easy. SIGGRAPH 2021 Real-Time Live Best-in-Show Link · Video
Ming-Yu Liu, Koki Nagano, Yeongho Seol, Rafael Valle, Jaewoo Seo, Ting-Chun Wang, Arun Mallya, Sameh Khamis, Wei Ping, Rohan Badlani, Kevin J. Shih, Bryan Catanzaro, Simon Yuen, Jan Kautz - RTMV: A Ray-Traced Multi-View Synthetic Dataset for Novel View Synthesis. ECCV 2022 Workshops arXiv · Project
Jonathan Tremblay, Moustafa Meshry, Alex Evans, Jan Kautz, Alexander Keller, Sameh Khamis, Thomas Müller, Charles Loop, Nathan Morrical, Koki Nagano, Towaki Takikawa, Stan Birchfield - Hierarchical Neural Implicit Pose Network for Animation and Motion Retargeting. arXiv 2021 arXiv · Project
Sourav Biswas, Kangxue Yin, Maria Shugrina, Sanja Fidler, Sameh Khamis
2020
- RePose: Learning Deep Kinematic Priors for Fast Human Pose Estimation. arXiv 2020 arXiv
Hossam Isack, Christian Haene, Cem Keskin, Sofien Bouaziz, Yuri Boykov, Shahram Izadi, Sameh Khamis
2019
- Neural Rerendering in the Wild. CVPR 2019 Oral · Best Paper Finalist arXiv · Video · Code · Project
Moustafa Meshry, Dan B. Goldman, Sameh Khamis, Hugues Hoppe, Rohit Pandey, Noah Snavely, Ricardo Martin-Brualla
2018
- Depth from Motion for Smartphone AR. SIGGRAPH Asia 2018 PDF
Julien Valentin, Adarsh Kowdle, Jonathan T. Barron, Neal Wadhwa, Max Dzitsiuk, Michael Schoenberg, Vivek Verma, Ambrus Csaszar, Eric Turner, Ivan Dryanovski, Joao Afonso, Jose Pascoal, Konstantine Tsotsos, Mira Leung, Mirko Schmidt, Onur Guleryuz, Sameh Khamis, Vladimir Tankovich, Sean Fanello, Shahram Izadi, Christoph Rhemann - The Need 4 Speed in Real-Time Dense Visual Tracking. SIGGRAPH Asia 2018 PDF
Adarsh Kowdle, Christoph Rhemann, Sean Fanello, Andrea Tagliasacchi, Jonathan Taylor, Philip Davidson, Mingsong Dou, Kaiwen Guo, Cem Keskin, Sameh Khamis, David Kim, Danhang Tang, Vladimir Tankovich, Julien Valentin, Shahram Izadi - LookinGood: Enhancing Performance Capture with Real-time Neural Re-Rendering. SIGGRAPH Asia 2018 arXiv · Video
Ricardo Martin-Brualla, Rohit Pandey, Shuoran Yang, Pavel Pidlypenskyi, Jonathan Taylor, Julien Valentin, Sameh Khamis, Philip Davidson, Anastasia Tkach, Peter Lincoln, Adarsh Kowdle, Christoph Rhemann, Dan B. Goldman, Cem Keskin, Steve Seitz, Shahram Izadi, Sean Fanello - ActiveStereoNet: End-to-End Self-Supervised Learning for Active Stereo Systems. ECCV 2018 Oral arXiv
Yinda Zhang, Sameh Khamis, Christoph Rhemann, Julien Valentin, Adarsh Kowdle, Vladimir Tankovich, Michael Schoenberg, Shahram Izadi, Thomas Funkhouser, Sean Fanello - StereoNet: Guided Hierarchical Refinement for Real-Time Edge-Aware Depth Prediction. ECCV 2018 arXiv
Sameh Khamis, Sean Fanello, Christoph Rhemann, Adarsh Kowdle, Julien Valentin, Shahram Izadi
2017
- Motion2Fusion: Real-time Volumetric Performance Capture. SIGGRAPH Asia 2017 PDF
Mingsong Dou, Philip Davidson, Sean Fanello, Sameh Khamis, Adarsh Kowdle, Christoph Rhemann, Vladimir Tankovich, Shahram Izadi
2016
- Holoportation: Virtual 3D Teleportation in Real-time. UIST 2016 PDF · Video
Sergio Orts-Escolano, Christoph Rhemann, Sean Fanello, Wayne Chang, Adarsh Kowdle, Yury Degtyarev, David Kim, Philip Davidson, Sameh Khamis, Mingsong Dou, Vladimir Tankovich, Charles Loop, Qin Cai, Philip Chou, Sarah Mennicken, Julien Valentin, Vivek Pradeep, Shenlong Wang, Sing Bing Kang, Pushmeet Kohli, Yuliya Lutchyn, Cem Keskin, Shahram Izadi - Fusion4D: Real-time Performance Capture of Challenging Scenes. SIGGRAPH 2016 PDF · Video
Mingsong Dou, Sameh Khamis, Yury Degtyarev, Philip Davidson, Sean Fanello, Adarsh Kowdle, Sergio Orts-Escolano, Christoph Rhemann, David Kim, Jonathan Taylor, Pushmeet Kohli, Vladimir Tankovich, Shahram Izadi - Efficient and Precise Interactive Hand Tracking through Joint, Continuous Optimization of Pose and Correspondences. SIGGRAPH 2016 PDF · Video
Jonathan Taylor, Lucas Bordeaux, Thomas Cashman, Bob Corish, Cem Keskin, Toby Sharp, Eduardo Soto, David Sweeney, Julien Valentin, Benjamin Luff, Arran Topalian, Erroll Wood, Sameh Khamis, Pushmeet Kohli, Shahram Izadi, Richard Banks, Andrew Fitzgibbon, Jamie Shotton - Fits Like a Glove: Rapid and Reliable Hand Shape Personalization. CVPR 2016 Spotlight PDF
David Joseph Tan, Thomas Cashman, Jonathan Taylor, Andrew Fitzgibbon, Daniel Tarlow, Sameh Khamis, Shahram Izadi, Jamie Shotton
2015
- Learning an Efficient Model of Hand Shape Variation from Depth Images. CVPR 2015 PDF · Supp · Video
Sameh Khamis, Jonathan Taylor, Jamie Shotton, Cem Keskin, Shahram Izadi, Andrew Fitzgibbon - Walking and Talking: A Bilinear Approach to Multi-Label Action Recognition. CVPRW 2015 PDF
Sameh Khamis, Larry S. Davis
2014
- CoConut: Co-Classification with Output Space Regularization. BMVC 2014 PDF · Poster
Sameh Khamis, Christoph H. Lampert - Joint Learning for Attribute-Consistent Person Re-Identification. ECCVW 2014 PDF
Sameh Khamis, Cheng-Hao Kuo, Vivek K. Singh, Vinay Shet, Larry S. Davis
2012 – 2013
- Collective Activity Detection using Hinge-loss Markov Random Fields. CVPRW 2013 PDF · Slides
Ben London, Sameh Khamis, Stephen H. Bach, Bert Huang, Lise Getoor, Larry S. Davis - Person Re-identification using Semantic Color Names and RankBoost. WACV 2013 PDF
Cheng-Hao Kuo, Sameh Khamis, Vinay Shet - Combining Per-Frame and Per-Track Cues for Multi-Person Action Recognition. ECCV 2012 PDF · Poster
Sameh Khamis, Vlad I. Morariu, Larry S. Davis - A Flow Model for Joint Action Recognition and Identity Maintenance. CVPR 2012 PDF · Poster · Spotlight
Sameh Khamis, Vlad I. Morariu, Larry S. Davis
Education
PhD, Computer Science · University of Maryland
2010 – 2015 · College Park, MD
- Advised by Larry Davis at UMIACS.
- Thesis: multi-person activity recognition from video sequences.
- Internships: Siemens Corporate Research, IST Austria (Lampert group), Microsoft Research Cambridge, SRI International.
MSc, Computer Science · University of Western Ontario
2008 – 2010 · London, Ontario
- Advised by Yuri Boykov.
- Thesis: semantic image segmentation.
- Wrote a parallel out-of-core push-relabel algorithm for massive regular graphs.