Real-time Video to Sign Language Video Synthesis

Table of Contents

Project Team Members

Dr. Nagarathna R

Senior Scientist, CSIR-4PI
Work Package Leader & Co-PI, CLAIM

Mr. Nallana Mithun Babu

Co-Work Package Leader & Co-PI, CLAIM

Dr. E Rajalakshmi

Scientist

Mr. Ajinkya Ashok Jagtap

Scientist

Mr. Manish Kumar

Scientist

Supporting Team (PAT / PA)

Niyantarana T - PAT II

T. Inish - PAT II

B. Arjun K C - PAT I

Araveti Arunkumar - PA II

Madduri Aiswarya Lakshmi - PAT I

M. Ragesh - PAT I

About Our SLP Models

We build various Sign Language Production (SLP) models that learn to predict the sign pose video sequence given the input gloss word sequence. We have built transformer and diffusion-based SLP models.

We identify the research gaps in SOTA SLP and have proposed a novel methodology that has better performance when compared to SOTA SLP approaches. The following are samples of generated sign language videos.