27 C
Jaipur
Thursday, October 22, 2020

Google Tests Sign Language Detector to Switch ‘Active Speaker’ in Video Calls

Must read

Acer Chromebook: Acer launches Chromebook Spin 513 and Halo smart speaker

Acer has launched its first Chromebook powered by the Qualcomm Snapdragon 7c compute platform – the Acer Chromebook Spin 513...

Huawei Mate 40: Huawei unveils the Mate 40 Pro and Pro+ smartphones

Huawei has bolstered its product line-up with the Mate 40 Series flagship smartphones--the Mate 40 Pro and the Mate 40 Pro+....

Flipkart: Flipkart, Reliance Jio deals push VC inflows to $3.6 billion in September quarter : Report – Latest News

Despite the pandemic-driven recession, venture capitalists continued to invest dollars into the country with September quarter seeing investment inflows more than doubling to USD...

google: Google has undermined competition, says South Korean antitrust chief – Latest News

South Korea's antitrust chief said on Thursday the agency believes US search engine giant Google has undermined competition, adding that the agency plans to...

With most of us holed up in our homes and coordinating work over video calls due to the COVID-19 pandemic, you might have become well-acquainted with the variety of video conferencing software. A great feature of these video calling apps is automatic switching between video feeds of the person talking in real-time. This, however, doesn’t work with sign language users and they could feel left out of the conversation.

Google researchers have decided to fix this accessibility issue by building a real-time sign language detection engine. It can detect when a person in a video call is trying to communicate using sign language and bring the spotlight on them. The engine will be able to tell when a person starts signing and make them the active speaker.

This model was presented by Google researchers at ECCV 2020. The research paper titled Real-Time Sign Language Detection using Human Pose Estimation talks about how a ‘plug and play’ detection engine was created for video conferencing apps. The efficiency and latency of the video feed were a crucial aspect and the new model can handle both very well. I mean, what good will a delayed and choppy video feed do?

Here’s a quick look at what the sign language engine sees in real-time:

Now, if you are wondering how this sign language detection engine works then Google has explained it all in detail. First, the video passes through PoseNet, which estimates the key points of the body such as eyes, nose, shoulders, and more. It helps the engine create a stick figure of the person and then compare its movements to a model trained with the German Sign Language corpus.

This is how the researchers detect that the person has started or stopped signing. But, how are they assigned an active speaker role when there is essentially no audio? That was one of the biggest hurdles and Google overcame it by building a web demo that transmits a 20kHz high-frequency audio signal to the video conferencing app you connect with it. This will fool the video conferencing app into thinking that the person using sign language is speaking and thus, make them an active speaker.

Google researchers have already managed to achieve 80% accuracy in predicting when a person starts signing. It can easily be optimized to reach over 90% accuracy, which is just amazing. This sign detection engine is just a demo (and a research paper) for now but it won’t be long until we see one of the popular video conferencing apps, be it Meet or Zoom, adopt this to make life easier for mute people.

Source link

- Advertisement -

More articles

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Latest article

Acer Chromebook: Acer launches Chromebook Spin 513 and Halo smart speaker

Acer has launched its first Chromebook powered by the Qualcomm Snapdragon 7c compute platform – the Acer Chromebook Spin 513...

Huawei Mate 40: Huawei unveils the Mate 40 Pro and Pro+ smartphones

Huawei has bolstered its product line-up with the Mate 40 Series flagship smartphones--the Mate 40 Pro and the Mate 40 Pro+....

Flipkart: Flipkart, Reliance Jio deals push VC inflows to $3.6 billion in September quarter : Report – Latest News

Despite the pandemic-driven recession, venture capitalists continued to invest dollars into the country with September quarter seeing investment inflows more than doubling to USD...

google: Google has undermined competition, says South Korean antitrust chief – Latest News

South Korea's antitrust chief said on Thursday the agency believes US search engine giant Google has undermined competition, adding that the agency plans to...

apple: Apple expands Express retail store format ahead of holiday season – Latest News

Apple is expanding a new physical store format as it tries to get iPhone 12 models into customers' hands, its retail chief told Reuters...