Visual Computing @ TCS Research

We at visual computing research group works on research problems related to synthesis, estimation, animation, and rendering of objects and scenes across space and time. Currently, we are focusing on building algorithms for,

automation in creative content creation using AI
interactive digital representation of physical environment
speech and emotion aware body gesture and facial expressions for 2D and 3D digital humans
re-rendered reality for Metaverse

To this end, we are solving various research problems such as, creating a 3D model of the surrounding world, building a personalized digital 3D avatar of humans including cloth and hair animation, 2D/3D conversational agents with expresive gestures and expressions, making 3D human avatars to interact with other objects in the 3D digital scene, and real-time rendering of the large sdynamic scene with the varying spatial lighting and reflectance.

Following are some of the selected publications.

Publications

	GarSim: Particle Based Neural Garment Simulator Authors: Lokender Tiwari, Brojeshwar Bhowmick IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023 (PDF and Project Page Coming Soon)
	Emotion-Controllable Generalized Talking Face Generation Authors: Sanjana Sinha, Sandika Biswas, Ravindra Yadav, Brojeshwar Bhowmick International Joint Conference on Artificial Intelligence (IJCAI), 2022 (Long Oral Presentation, <4% of the papers are long oral accept)
	Talk-to-Resolve: Combining scene understanding and spatial dialogue to resolve granular task ambiguity for a collocated robot Authors: Pradip Pramanick, Chayan Sarkar, Snehasis Banerjee, Brojeshwar Bhowmick Journal of Robotics and Autonomous Systems (RAS), 2022
	Realistic Talking Face Animation with Speech-Induced Head Motion Authors: Sandika Biswas, Sanjana Sinha, Dipanjan Das, Brojeshwar Bhowmick Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP), 2021 (Oral Presentation) (Best Paper Award) Media Mention: Techxplore
	DeepDraper: Fast and Accurate 3D Garment Draping over a 3D Human Body Authors: Lokender Tiwari, Brojeshwar Bhowmick Workshop in International Conference on Computer Vision (ICCV), 2021 Media Mention: Techxplore \| NDTVGadgets360 \| BusinessAPAC \| supplementary \|
	3DPoseLite: A Compact 3D Pose Estimation Using Node Embeddings Authors: Meghal Dani, Karan Narain, Ramya Hebbalaguppe IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2021
	Speech-driven Facial Animation using Cascaded GANs for Learning of Motion and Texture Authors: Dipanjan Das, Sandika Biswas, Sanjana Sinha, Brojeshwar Bhowmick European Conference on Computer Vision (ECCV), 2020
	PoseFromGraph: Compact 3-D Pose Estimation using Graphs Authors: Meghal Dani, Additya Popli, Ramya Hebbalaguppe SIGGRAPH Technical Communication , 2020 Project Page
	SmartOverlays: A Visual Saliency Driven Label Placement for Intelligent Human-Computer Interfaces Authors: Srinidhi Hegde, Jitender Maurya, Ramya Hebbalaguppe, Aniruddha Kalkar IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2020 Project Page
	Realistic Lip Animation from Speech for Unseen Subjects using Few-shot Cross-modal Learning Authors: Swapna Agarwal, Dipanjan Das, Brojeshwar Bhowmick European Signal Processing Conference (EUSIPCO), 2020
	MOMBAT: Improving Face Video based Heart Rate Monitoring using Modeling and Tracking Authors: Puneet Gupta, Brojeshwar Bhowmick, Arpan pal Journal of Computers in Biology and Medicine, 2020
	Identity-Preserving Realistic Talking Face Generation Authors: Sanjana Sinha, Sandika Biswas, Brojeshwar Bhowmick International Joint Conference on Neural Networks (IJCNN), 2020
	Variational Clustering: Leveraging Variational Autoencoders for Image Clustering Authors: Vignesh Prasad, Dipanjan Das, Brojeshwar Bhowmick International Joint Conference on Neural Networks (IJCNN), 2020
	Talk to the Vehicle: Language Conditioned Autonomous Navigation of Self Driving Cars Authors: Sriram N. N., Tirth Maniar, Jayaganesh Kalyanasundaram, Vineet Gandhi, Brojeshwar Bhowmick, K. Madhava Krishna International Conference on Intelligent Robots and Systems (IROS), 2019
	Lifting 2d Human Pose to 3d : A Weakly Supervised Approach Authors: Sandika Biswas, Sanjana Sinha, Kavya Gupta, Brojeshwar Bhowmick International Joint Conference on Neural Networks (IJCNN), 2019 (Oral Presentation)
	Multi-Modal Image Stitching With Non-linear Optimization Authors: Arindam Saha, Soumyadip Maity, Brojeshwar Bhowmick International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2019 \| poster \|
	SfMLearner++: Learning Monocular Depth & Ego-Motion using Meaningful Geometric Constraints Authors: Vignesh Prasad, Brojeshwar Bhowmick Winter Conference on Applications of Computer Vision (WACV), 2019
	Deep Representation Learning Characterized by Inter-class Separation for Image Clustering Authors: Dipanjan Das, Ratul Ghosh, Brojeshwar Bhowmick Winter Conference on Applications of Computer Vision (WACV), 2019
	Epipolar Geometry based Learning of Multi-view Depth and Ego-Motion from Monocular Sequences Authors: Vignesh Prasad, Dipanjan Das, Brojeshwar Bhowmick Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP), 2018 (Oral Presentation) (Best paper award) Media Mention: Techxplore
	Indoor Dense Depth Map at Drone Hovering Authors: Arindam Saha, Soumyadip Maity, Brojeshwar Bhowmick International Conference on Image Processing (ICIP), 2018
	Exploring the Feasibility of Face video based Instantaneous Heart-rate for Micro-expression Spotting Authors: Puneet Gupta, Brojeshwar Bhowmick, Arpan Pal Workshop in International Conference on Computer Vision and Pattern Recogntion (CVPR), 2018
	Edge SLAM: Edge Points Based Monocular Visual SLAM Authors: Soumyadip Maity, Arindam Saha, Brojeshwar Bhowmick Workshop in International Conference on Computer Vision (ICCV), 2017
	3D Point Cloud Registration With Shape Constraint Authors: Swapna Agarwal, Brojeshwar Bhowmick International Conference on Image Processing(ICIP), 2017
	MOTION BLUR REMOVAL VIA COUPLED AUTOENCODER Authors: Kavya Gupta, Brojeshwar Bhowmick, Angshul Majumdar, International Conference on Image Processing(ICIP), 2017
	ACCURATE HEART-RATE ESTIMATION FROM FACE VIDEOS USING QUALITY-BASED FUSION Authors: Puneet Gupta, Brojeshwar Bhowmick, Arpan Pal, International Conference on Image Processing(ICIP), 2017
	Divide and Conquer: A Hierarchical Approach to Large-scale Structure-from-Motion Authors: Brojeshwar Bhowmick, Suvam Patra, Avishek Chatterjee, Venu Madhav Govindu, Subhashis Banerjee, Journal of Computer Vision and Image Understanding(CVIU), 2017
	Accurate estimation of joint motion trajectories for rehabilitation using Kinect Authors: Sanjana Sinha, Brojeshwar Bhowmick, Aniruddha Sinha, Abhijit Das International Conference of the IEEE Engineering in Medicine and Biology Society(EMBC), 2017
	Serial fusion of Eulerian and Lagrangian approaches for accurate heart-rate estimation using face videos Authors: Puneet Gupta, Brojeshwar Bhowmick, Arpan Pal International Conference of the IEEE Engineering in Medicine and Biology Society(EMBC), 2017
	An efficient and robust method of virtual augmentation of eye-glass for easy shopping Authors: Apurbaa Mallik, Brojeshwar Bhowmick International Conference on Control, Automation, Robotics and Vision(ICARCV), 2016
	Divide and Conquer: Efficient large-scale structure from motion using graph partitioning Authors: Brojeshwar Bhowmick, Suvam Patra, Avishek Chatterjee, Venu Madhav Govindu, Subhashis Banerjee Asian Conference on Computer Vision(ACCV), 2014
	Mobiscan3D: A low cost framework for real time dense 3D reconstruction on mobile devices Authors: Brojeshwar Bhowmick, Apurbaa Mallik, Arindam Saha International Conference on Ubiquitous Intelligence & Computing(UIC), 2014
	Person identification using skeleton information from kinect Authors: Aniruddha Sinha, Kingshuk Chakravarty, Brojeshwar Bhowmick International Conference on Advances in Computer-Human Interactions(ACHI), 2013
	High Resolution Point Cloud Generation from Kinect and HD Cameras using Graph Cut Authors: Suvam Patra, Brojeshwar Bhowmick, Subhashis Banerjee, Prem Kalra International Conference on Computer Vision Theory and Application(VISSAP), 2012
	A Kalman Filter Based Approach to De-noise the Stereo Vision Based Pedestrian Position Estimation Authors: Arijit Sinharay, Arpan Pal, Brojeshwar Bhowmick International Conference on Computer Modelling and Simulation(UKSIM), 2011
	Stereo Vision Based Pedestrians Detection and Distance Measurement for Automotive Application Authors: Brojeshwar Bhowmick, Sambit Bhadra, Arijit Sinharay International Conference on Intelligent Systems, Modelling and Simulation(ISMS), 2011

Visual Computing @ TCS Research

Publications

GarSim: Particle Based Neural Garment Simulator

Emotion-Controllable Generalized Talking Face Generation

Talk-to-Resolve: Combining scene understanding and spatial dialogue to resolve granular task ambiguity for a collocated robot

Realistic Talking Face Animation with Speech-Induced Head Motion

DeepDraper: Fast and Accurate 3D Garment Draping over a 3D Human Body

3DPoseLite: A Compact 3D Pose Estimation Using Node Embeddings

Speech-driven Facial Animation using Cascaded GANs for Learning of Motion and Texture

PoseFromGraph: Compact 3-D Pose Estimation using Graphs

SmartOverlays: A Visual Saliency Driven Label Placement for Intelligent Human-Computer Interfaces

Realistic Lip Animation from Speech for Unseen Subjects using Few-shot Cross-modal Learning

MOMBAT: Improving Face Video based Heart Rate Monitoring using Modeling and Tracking

Identity-Preserving Realistic Talking Face Generation

Variational Clustering: Leveraging Variational Autoencoders for Image Clustering

Talk to the Vehicle: Language Conditioned Autonomous Navigation of Self Driving Cars

Lifting 2d Human Pose to 3d : A Weakly Supervised Approach

Multi-Modal Image Stitching With Non-linear Optimization

SfMLearner++: Learning Monocular Depth & Ego-Motion using Meaningful Geometric Constraints

Deep Representation Learning Characterized by Inter-class Separation for Image Clustering

Epipolar Geometry based Learning of Multi-view Depth and Ego-Motion from Monocular Sequences

Indoor Dense Depth Map at Drone Hovering

Exploring the Feasibility of Face video based Instantaneous Heart-rate for Micro-expression Spotting

Edge SLAM: Edge Points Based Monocular Visual SLAM

3D Point Cloud Registration With Shape Constraint

MOTION BLUR REMOVAL VIA COUPLED AUTOENCODER

ACCURATE HEART-RATE ESTIMATION FROM FACE VIDEOS USING QUALITY-BASED FUSION

Divide and Conquer: A Hierarchical Approach to Large-scale Structure-from-Motion

Accurate estimation of joint motion trajectories for rehabilitation using Kinect

Serial fusion of Eulerian and Lagrangian approaches for accurate heart-rate estimation using face videos

An efficient and robust method of virtual augmentation of eye-glass for easy shopping

Divide and Conquer: Efficient large-scale structure from motion using graph partitioning

Mobiscan3D: A low cost framework for real time dense 3D reconstruction on mobile devices

Person identification using skeleton information from kinect

High Resolution Point Cloud Generation from Kinect and HD Cameras using Graph Cut

A Kalman Filter Based Approach to De-noise the Stereo Vision Based Pedestrian Position Estimation

Stereo Vision Based Pedestrians Detection and Distance Measurement for Automotive Application