kalyan sunkavalli     

Kalyan Sunkavalli

I am a Principal Scientist at Adobe Research. I received my Ph.D. from Harvard University in 2012 under the supervision of Hanspeter Pfister. I received my Masters in 2006 from Columbia University where I was advised by Ravi Ramamoorthi. I graduated from the Indian Institute of Technology, Delhi in 2003.

My research interests lie at the intersection of graphics, computer vision and machine learning; in particular, my work focuses on reconstructing different aspects of visual appearance in images—illumination, geometry, reflectance, color, texture, motion, etc.—and building tools that enable users to edit and enhance them.

Internships at Adobe: I am always looking for motivated graduate students to collaborate with. If you are interested in interning at Adobe with me, send me an email describing your research interests.

CV | Google Scholar | LinkedIn

Publications


     MatFormer: A Generative Model for Procedural Materials
Paul Guerrero, Miloš Hašan, Kalyan Sunkavalli, Radomír Měch, Tamy Boubekeur, Niloy J. Mitra
SIGGRAPH 2022
 
     NeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstruction
Xiaoshuai Zhang, Sai Bi, Kalyan Sunkavalli, Hao Su, Zexiang Xu
CVPR 2022 (oral)
arxiv | project
 
     Point-NeRF: Point-based Neural Radiance Fields
Qiangeng Xu, Zexiang Xu, Julien Philip, Sai Bi, Zhixin Shu, Kalyan Sunkavalli, Ulrich Neumann
CVPR 2022 (oral)
arxiv | project | code
 
     PhotoScene: Photorealistic Material and Lighting Transfer for Indoor Scenes
Yu-Ying Yeh, Zhengqin Li, Yannick Hold-Geoffroy, Rui Zhu, Zexiang Xu, Miloš Hašan, Kalyan Sunkavalli, Manmohan Chandraker
CVPR 2022
paper | project
 
     RigNeRF: Fully Controllable Neural 3D Portraits
ShahRukh Athar, Zexiang Xu, Kalyan Sunkavalli, Eli Shechtman, Zhixin Shu
CVPR 2022
 
     Material Swapping for 3D Scenes using a Learnt Material Similarity Measure
Maxine Perroni-Scharf, Kalyan Sunkavalli, Jonathan Eisenmann, Jonathan Eisenmann
WiCV Workshop, CVPR 2022
 
 
     SSH: A Self-Supervised Framework for Image Harmonization
Yifan Jiang, He Zhang, Jianming Zhang, Yilin Wang, Zhe Lin, Kalyan Sunkavalli, Simon Chen, Sohrab Amirghodsi, Sarah Kong, Zhangyang Wang
ICCV 2021
arxiv | code
 
     NeuTex: Neural Texture Mapping for Volumetric Neural Rendering
Fanbo Xiang, Zexiang Xu, Miloš Hašan, Yannick Hold-Geoffroy, Kalyan Sunkavalli, Hao Su
CVPR 2021 (oral)
arxiv | project | code
 
     OpenRooms: An Open Framework for Photorealistic Indoor Scene Datasets
Zhengqin Li, Ting-Wei Yu, Shen Sang, Sarah Wang, Meng Song, Yuhan Liu, Yu-Ying Yeh, Rui Zhu, Nitesh Gundavarapu
Jia Shi, Sai Bi, Zexiang Xu, Hong-Xing Yu, Kalyan Sunkavalli, Miloš Hašan, Ravi Ramamoorthi, Manmohan Chandraker
CVPR 2021 (oral)
arxiv | project
 
     Deep Denoising of Flash and No-Flash Pairs for Photography in Low-Light Environments
Zhihao Xia, Michaël Gharbi, Federico Perazzi, Kalyan Sunkavalli, Ayan Chakrabarti
CVPR 2021
arxiv | project
 
     MATch: Differentiable Material Graphs for Procedural Material Capture
Liang Shi, Beichen Li, Miloš Hašan, Kalyan Sunkavalli, Radomir Mech, Tamy Boubekeur, Wojciech Matusik
SIGGRAPH Asia 2020
project
 
     MaterialGAN: Reflectance Capture using a Generative SVBRDF Model
Yu Guo, Cameron Smith, Miloš Hašan, Kalyan Sunkavalli, Shuang Zhao
SIGGRAPH Asia 2020
arxiv | project
 
     Neural Reflectance Fields for Appearance Acquisition
Sai Bi*, Zexiang Xu*, Pratul Srinivasan, Ben Mildenhall, Kalyan Sunkavalli, Miloš Hašan, Yannick Hold-Geoffroy, David Kriegman, Ravi Ramamoorthi
arxiv
 
     Deep Reflectance Volumes: Relightable Reconstructions from Multi-View Photometric Images
Sai Bi, Zexiang Xu, Kalyan Sunkavalli, Miloš Hašan, Yannick Hold-Geoffroy, David Kriegman, Ravi Ramamoorthi
ECCV 2020 (spotlight)
arxiv
 
     Single View Metrology in the Wild
Rui Zhu, Xingyi Yang, Yannick Hold-Geoffroy, Federico Perazzi, Jonathan Eisenmann, Kalyan Sunkavalli, Manmohan Chandraker
ECCV 2020
arxiv | project
 
     Deep Multi Depth Panoramas for View Synthesis
Kai-En Lin, Zexiang Xu, Ben Mildenhall, Pratul P. Srinivasan, Yannick Hold-Geoffroy, Stephen DiVerdi, Qi Sun, Kalyan Sunkavalli, Ravi Ramamoorthi
ECCV 2020
arxiv
 
     State of the Art on Neural Rendering
Ayush Tewari*, Ohad Fried*, Justus Thies*, Vincent Sitzmann*, Stephen Lombardi, Kalyan Sunkavalli, Ricardo Martin-Brualla, Tomas Simon, Jason Saragih, Matthias Nießner, Rohit Pandey, Sean Fanello, Gordon Wetzstein, Jun-Yan Zhu, Christian Theobalt, Maneesh Agrawala, Eli Shechtman, Dan B. Goldman and Michael Zollhöfer
CGF 2020
arxiv | CVPR 2020 Tutorial on Neural Rendering
 
     Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Images
Sai Bi, Zexiang Xu, Kalyan Sunkavalli, David Kriegman, Ravi Ramamoorthi
CVPR 2020
arxiv | video
 
     Basis Prediction Networks for Effective Burst Denoising with Large Kernels
Zhihao Xia, Federico Perazzi, Michaël Gharbi, Kalyan Sunkavalli, Ayan Chakrabarti
CVPR 2020
arxiv | project
 
     Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image
Zhengqin Li, Mohammad Shafiei, Ravi Ramamoorthi, Kalyan Sunkavalli, Manmohan Chandraker
CVPR 2020 (oral)
arxiv | project
 
     Learning Monocular Face Reconstruction using Multi-View Supervision
Zhixin Shu, Duygu Ceylan, Kalyan Sunkavalli, Eli Shechtman, Sunil Hadap, Dimitris Samaras
FG 2020
paper
 
     Self-supervised Multi-view Person Association and Its Applications
Minh Vo, Ersin Yumer, Kalyan Sunkavalli, Sunil Hadap, Yaser Sheikh, Srinivasa Narasimhan
TPAMI 2020
arxiv | project | paper | video
 
     Synthetic to Real Translation Via Explicit Image Disentanglement
Sai Bi, Kalyan Sunkavalli, Federico Perazzi, Eli Shechtman, Vladimir Kim, Ravi Ramamoorthi
ICCV 2019
arxiv | paper | supplementary
 
     Deep Parametric Indoor Lighting Estimation
Marc-André Gardner, Yannick Hold-Geoffroy, Kalyan Sunkavalli, Christian Gagné, Jean-Francois Lalonde
ICCV 2019
arxiv | project
 
     Deep Single-Image Portrait Relighting
Hao Zhou, Sunil Hadap, Kalyan Sunkavalli, David Jacobs
ICCV 2019
project
 
     Deep View Synthesis from Sparse Photometric Images
Zexiang Xu, Sai Bi, Kalyan Sunkavalli, Sunil Hadap, Hao Su, Ravi Ramamoorthi
SIGGRAPH 2019
paper | supplementary | video
 
     Fast Spatially-Varying Indoor Lighting Estimation
Mathieu Garon, Kalyan Sunkavalli, Nathan Carr, Sunil Hadap, Jean-Francois Lalonde
CVPR 2019 (oral)
arxiv | project | paper | supplementary | data
 
     Learning to Separate Multiple Illuminants in a Single Image
Zhuo Hui, Ayan Chakrabarti, Kalyan Sunkavalli, Aswin Sankaranarayanan
CVPR 2019
arxiv | project | paper | supplementary | video
 
     All-Weather Deep Outdoor Lighting Estimation
CVPR 2019
Jinsong Zhang, Kalyan Sunkavalli, Yannick Hold-Geoffroy, Sunil Hadap, Jonathan Eisenmann, Jean-Francois Lalonde arxiv | project | paper | supplementary
 
     Learning to Reconstruct Shape and Spatially-varying Reflectance from a Single Image
Zhengqin Li, Zexiang, Xu, Ravi Ramamoorthi, Kalyan Sunkavalli, Manmohan Chandraker
SIGGRAPH Asia 2018
arxiv | project | paper | video
 
     Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image
Zhengqin Li, Kalyan Sunkavalli, Manmohan Chandraker
ECCV 2018 (oral)
arxiv | paper | supplementary
 
     MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics
Xinchen Yan, Akash Rastogi, Ruben Villegas, Kalyan Sunkavalli, Eli Shechtman, Sunil Hadap, Ersin Yumer, Honglak Lee
ECCV 2018
arxiv | project | paper | code
 
     Compositing-aware Image Search
Hengshuang Zhao, Xiaohui Shen, Zhe Lin, Kalyan Sunkavalli, Brian Price, Jiaya Jia
ECCV 2018
project | paper
 
     Deep Image-based Relighting using Optimal Sparse Samples
Zexiang Xu, Kalyan Sunkavalli, Sunil Hadap, Ravi Ramamoorthi
SIGGRAPH 2018
project | paper | video | code
 
     Illuminant Spectra-based Source Separation Using Flash Photography
Zhuo Hui, Kalyan Sunkavalli, Sunil Hadap, Aswin C. Sankaranarayanan
CVPR 2018 (oral)
arxiv | project | paper | video | supplementary | code
 
     A Perceptual Measure for Deep Single Image Camera Calibration
Yannick Hold-Geoffroy, Kalyan Sunkavalli, Jonathan Eisenmann, Matthew Fisher, Emiliano Gambaretto, Sunil Hadap, Jean-François Lalonde
CVPR 2018
arxiv | project | paper | supplementary
 
     Fast Video Object Segmentation by Reference-Guided Mask Propagation
Seoung Wug Oh, Joon-Young Lee, Kalyan Sunkavalli, Seon Joo Kim
CVPR 2018 (spotlight)
paper | code
 
     Learning to Predict Indoor Illumination from a Single Image
Marc-Andre Gardner, Kalyan Sunkavalli, Ersin Yumer, Xiaohui Shen, Emiliano Gambaretto, Christian Gagné, Jean-François Lalonde
SIGGRAPH Asia 2017
arxiv | project | paper | supplementary | data
 
     Reflectance Capture using Univariate Sampling of BRDFs
Zhuo Hui, Kalyan Sunkavalli, Joon-Young Lee, Sunil Hadap, Jian Wang, Aswin C. Sankaranarayanan
ICCV 2017
paper | video
 
     Scene Parsing with Global Context Embedding
Wei-Chih Hung, Yi-Hsuan Tsai, Xiaohui Shen, Zhe Lin, Kalyan Sunkavalli, Xin Lu, Ming-Hsuan Yang
ICCV 2017
arxiv | code
 
     Photometric Stabilization for Fast-forward videos
Xuaner Zhang, Joon-Young Lee, Kalyan Sunkavalli, Zhaowen Wang
Pacific Graphics 2017
arxiv | paper
 
 
     Nautilus: Recovering Regional Symmetry Transformations for Image Editing
Michal Lukáč, Daniel Sýkora, Kalyan Sunkavalli, Eli Shechtman, Ondřej Jamriška, Nathan Carr, Tomáš Pajdla
SIGGRAPH 2017
project | paper
 
     Portrait Lighting Transfer using a Mass Transport Approach
Zhixin Shu, Sunil Hadap, Eli Shechtman, Kalyan Sunkavalli, Sylvain Paris, Dimitris Samaras
TOG 2017
project | paper | supplementary
 
     Deep Outdoor Illumination Estimation
Yannick Hold-Geoffroy, Kalyan Sunkavalli, Sunil Hadap, Emiliano Gambaretto, Jean-François Lalonde
CVPR 2017 (oral)
arxiv | project | paper | supplementary | demo
 
     Neural Face Editing with Intrinsic Image Disentangling
Zhixin Shu, Ersin Yumer, Sunil Hadap, Kalyan Sunkavalli, Eli Shechtman, Dimitris Samaras
CVPR 2017 (oral)
arxiv | project | paper
 
     Deep Image Harmonization
Yi-Hsuan Tsai, Xiaohui Shen, Zhe Lin, Kalyan Sunkavalli, Xin Lu, Ming-Hsuan Yang
CVPR 2017
arxiv | paper | code and data
 
     Consistent Video Filtering for Camera Arrays
Nicolas Bonneel, James Tompkin, Deqing Sun, Oliver Wang, Kalyan Sunkavalli, Sylvain Paris, Hanspeter Pfister
Eurographics 2017
project | paper | supplementary
 
     Single-image RGB Photometric Stereo With Spatially-varying Albedo
Ayan Chakrabarti, Kalyan Sunkavalli
3DV 2016 (oral)
arxiv | project | paper | code and data
 
     Removing Shadows from Images of Documents
Steve Bako, Soheil Darabi, Eli Shechtman, Jue Wang, Kalyan Sunkavalli, Pradeep Sen
ACCV 2016
project | paper
 
     Appearance Harmonization for Single Image Shadow Removal
Li-Qian Ma, Jue Wang, Eli Shechtman, Kalyan Sunkavalli, Shi-Min Hu
Pacific Graphics 2016
paper
 
     Shading-aware Multi-view Stereo
Fabian Langguth, Kalyan Sunkavalli, Sunil Hadap, Michael Goesele
ECCV 2016
project | paper | supplementary | code
 
     Sky is Not the Limit: Semantic-Aware Sky Replacement
Yi-Hsuan Tsai, Xiaohui Shen, Zhe Lin, Kalyan Sunkavalli, Ming-Hsuan Yang
SIGGRAPH 2016
project | paper | supplementary

This work was demoed at Adobe MAX 2015 Sneaks as Sky Replace.
 
     White Balance under Mixed Illumination using Flash Photography
Zhuo Hui, Aswin Sankaranarayanan, Kalyan Sunkavalli, Sunil Hadap
ICCP 2016
project | paper | code and data
 
     Automatic Content-Aware Color and Tone Stylization
Joon-Young Lee, Kalyan Sunkavalli, Zhe Lin, Xiaohui Shen, In So Kweon
CVPR 2016 (spotlight)
arxiv | paper | data

This is the technology behind the Smart Looks feature in Photoshop Elements 14.
 
     PatchMatch-based Automatic Lattice Detection for Near-Regular Textures
Siying Liu, Tian-Tsong Ng, Kalyan Sunkavalli, Minh Do, Eli Shechtman, Nathan Carr
ICCV 2015
paper | supplementary
 
     Blind Video Temporal Consistency
Nicolas Bonneel, James Tompkin, Kalyan Sunkavalli, Deqing Sun, Sylvain Paris, Hanspeter Pfister
SIGGRAPH Asia 2015
project | paper | video
 
     High-Quality Hair Modeling from A Single Portrait Photo
Mengelei Chai, Linjie Luo, Kalyan Sunkavalli, Nathan Carr, Sunil Hadap, Kun Zhou
SIGGRAPH Asia 2015
paper | video

This work was demoed at Adobe MAX 2015 Sneaks as 3D Portraits.
 
     Smooth Loops for Unconstrained Video
Laura Sevilla-Lara, Jonas Wulff, Kalyan Sunkavalli, Eli Shechtman
EGSR 2015
project | paper | video
 
     Interactive Intrinsic Video Editing
Nicolas Bonneel, Kalyan Sunkavalli, James Tompkin, Deqing Sun, Sylvain Paris, Hanspeter Pfister
SIGGRAPH Asia 2014
project | paper | video
 
     Time-lapse Photometric Stereo and Applications
Fangyang Shen, Kalyan Sunkavalli, Nicolas Bonneel, Szymon Rusinkiewicz, Hanspeter Pfister, Xin Tong
Pacific Graphics 2014
project | paper
 
     Automatic scene inference for 3D object compositing
Kevin Karsch, Kalyan Sunkavalli, Sunil Hadap, Nathan Carr, Hailin Jin, Rafael Fonte, Michael Sittig, David Forsyth
TOG 2014
project | paper | video | supplementary

We demoed our work as part of Adobe MAX 2014 Sneaks.
 
     Example-based Video Color Grading
Nicolas Bonneel, Kalyan Sunkavalli, Sylvain Paris, Hanspeter Pfister
SIGGRAPH 2013
project | paper | video | code

Our work was demoed at Adobe MAX 2013 Sneaks.
 
     Video Snapshots: Creating High-Quality Images from Video Clips
Kalyan Sunkavalli, Neel Joshi, Sing Bing Kang, Michael F. Cohen, Hanspeter Pfister
TVCG 2012
project | paper | supplementary
 
     Video Face Replacement
Kevin Dale, Kalyan Sunkavalli, Micah K. Johnson, Daniel Vlasic, Wojciech Matusik, Hanspeter Pfister
SIGGRAPH Asia 2011
project | paper | video

Our work was featured in the SIGGRAPH Asia 2011 Technical Papers trailer and has received some press coverage.
 
     Visibility Subspaces: Uncalibrated Photometric Stereo with Shadows
Kalyan Sunkavalli, Todd Zickler, Hanspeter Pfister
ECCV 2010
project | paper | poster | presentation | data
 
     Multi-scale Image Harmonization
Kalyan Sunkavalli, Micah K. Johnson, Wojciech Matusik, Hanspeter Pfister
SIGGRAPH 2010
project | paper | supplementary | presentation | code | data

Our work was featured on the inside cover of the Proceedings of SIGGRAPH 2010 and on the highlights of SIGGRAPH 2010.
 
     Image Restoration using Online Photo Collections
Kevin Dale, Micah K. Johnson, Kalyan Sunkavalli, Wojciech Matusik, Hanspeter Pfister
ICCV 2009
project | paper
 
     What do color changes reveal about an outdoor scene?
Kalyan Sunkavalli, Fabiano Romeiro, Wojciech Matusik, Todd Zickler, Hanspeter Pfister
CVPR 2008 (oral)
project | paper | video | presentation | data
 
     Factored Time-Lapse Video
Kalyan Sunkavalli, Wojciech Matusik, Hanspeter Pfister, Szymon Rusinkiewicz
SIGGRAPH 2007
project | paper | video | presentation | data
 
     Time-Varying BRDFs
Bo Sun, Kalyan Sunkavalli, Ravi Ramamoorthi, Peter Belhumeur, Shree Nayar
TVCG 2007
project | paper | video | presentation | data
 
     First steps toward an electronic field guide for plants
Gaurav Agarwal, Peter Belhumeur, Steven Feiner, David Jacobs, W. John Kress, Ravi Ramamoorthi, Norman A. Bourg, Nandan Dixit, Haibin Ling, Dhruv Mahajan, Rusty Russell, Sameer Shirdhonkar, Kalyan Sunkavalli, Sean White
Taxon 2006
paper
 
     Parsing News Video Using Integrated Audio-video Features
S. Kalyan Krishna, Raghav Subbarao, Santanu Chaudhury, Arun Kumar
PReMI 2005
paper