Interact with ChatGPT from your thumb
One of my favourite things to do is tinkering with different interaction models, hence the name SoundxVision. This time, I've been playing around with using voice as an input method, specifically how to capture a user's intention to communicate with a #virtualassistant.
Usually, waking up a virtual assistant requires the user to say a wake-up phrase like "Hey Siri," "OK Google," or "Alexa" (which, let's be honest, is the best one so far). But it's not always reliable or appropriate to speak out loud, not to mention that I've accidentally triggered Siri so many times, and it's super annoying. Another issue is ending the prompt or command. Often, the virtual assistant ends the prompt before users finished speaking, or it takes too long to detect the end of of the command. And since users are often very polite, they usually wait for Siri to "Uhm" or "Uh huh" before they start speaking, but with a gesture-based approach, they could start speaking right away without feeling impolite.
Usually, waking up a virtual assistant requires the user to say a wake-up phrase like "Hey Siri," "OK Google," or "Alexa" (which, let's be honest, is the best one so far). But it's not always reliable or appropriate to speak out loud, not to mention that I've accidentally triggered Siri so many times, and it's super annoying. Another issue is ending the prompt or command. Often, the virtual assistant ends the prompt before users finished speaking, or it takes too long to detect the end of of the command. And since users are often very polite, they usually wait for Siri to "Uhm" or "Uh huh" before they start speaking, but with a gesture-based approach, they could start speaking right away without feeling impolite.
So, we came up with a solution, “raise hand to talk”, based on gesture recognition on our thumb ring, to activate a virtual assistant or #chatbot (yep, we are talking about #ChatGPT #AI chatbot in this demo) that uses speech as an input. All you gotta do is raise your hand in front of your mouth like you're holding a microphone - easy peasy! This gesture is simple to learn, but distinct enough so it doesn't get misrecognized. We've been able to catch the user's intention most of the time, and we're constantly improving it.
Demo of the gesture based approach to wake up and interact with ChatGPT using Whisper speech to text API
Just picture it - it's like having a tiny microphone on your finger that only activates when you want it to. It's relatable too, 'cause we're all used to using microphones to speak, right? So now you can feel like a rockstar while interacting with your virtual assistant or chatbot. It's all about making tech feel fun and intuitive! 🎤😄
And guess what? With the microphone so close to your mouth that the audio is superb for speech recognition! It's so good that we're even experimenting with silent speech recognition. Now on, you won't have to worry about waking up your partner at night or disturbing a meeting while trying to find an answer to a question that came up earlier (in the same meeting). It's all about keeping things smooth and convenient, while also being considerate of those around you. Just raise your hand and let the magic happen! 🎤🔥😴🚀
The best use case for this is, as always, with #AR #VR devices, especially with those small ones for outdoor usage, we will soon have demos for those systems.
This demo used OpenAI #Whisper API for speech recognition and #ChatGPT for responding to the user commands.
The best use case for this is, as always, with #AR #VR devices, especially with those small ones for outdoor usage, we will soon have demos for those systems.
This demo used OpenAI #Whisper API for speech recognition and #ChatGPT for responding to the user commands.