Swiping in XR with a Thumb Ring
Swiping is the most used and powerful when it comes to #interaction in the #smartphones era, and its intuitiveness is the key element that makes it so popular. The gesture is peanut simple, moving the finger left, right, up, down for manipulating #graphicalUserInterface makes so much senses and super powerful when consider the swiping speed, angle varies and can be take into account for endless ways of content manipulation. The swiping gesture is so seamless that often users do not aware that many of our everyday applications can be used with just few simple swipe, Tinder with it recognisable swipe right, MobilePay emphasised swiping as a fast and secure way for payment, TikTok can be used with single swiping direction, Instagram or Facebook are also utilised swiping as the main way of interactions, the list goes on. And for XR to be widely adopted, I have no doubt that it should be our main way of interact. BUT,
Swiping in XR
There is a big BUT, while swiping has become a popular and intuitive way of interacting with smartphones, it can be cumbersome in XR devices, requiring awkward hand positions or excessive movements (with computer vision based hand tracking), traditional controller with physical thumbstick works but I highly doubt it is appropriate for everyday and outdoor usage, that's when wearables comes into the game to unlock the full potential of swiping on XR devices. In my opinion, there are 5 requirements that needs to be fulfilled for the swiping wearable to check the box of effortless interaction:
- Minimal movements: just like swiping on smartphones, the finger movements distance should not excess the size of the typical smartphone screen.
- Support at least horizontal and vertical swiping.
- To be worn like a ring (on the finger's lowest phalanx).
- Decent swiping angle: too small of an swiping angle can make it difficult to swipe accurately, at an appropriate amount of swiping angle, it can also be used as a variable for manipulating graphical contents dynamically.
- Responsive: the object manipulation needs to be inlined with the physical movement.
A SoundxVision thumb ring prototype for swiping with XR devices
With those in mind, I would like to present SoundxVision ring prototype that I created in order to reach the goal of all-day input device for XR. To put this simple, this ring senses thumb gestures for horizontal and vertical movement, the gesture is recognised when the thumb is in contact and move across the index finger, which as the result, the swiping angle can be up to 60 degrees, horizontally and vertically. The ring uses Bluetooth Low Energy to communicate with XR devices. The unique aspect about the gestures recognition with this prototype is users can swipe horizontally and vertically with the same movement, for vertical swiping, just align the index finger vertically, this mapping makes sense and can be easily adopted.
In this video, a demo was made for Meta Oculus VR Quest 2, on the demo #1, a concept of a social network application was presented with swiping priority, the demo #2 is simple interaction with virtual objects, in this case changing the car appearance with just a swipe, at the end of the video is a closer looks on the recognised gestures and its responsiveness.
There are already input ring with touch sensors (capacitive for example, some even has a joystick but the size on those are not at all joyful), but the touch surface on those are small, so the swiping throw/angle, as the result tiny, which may not feels good or be accurate, and due to the size limitation it usually be the case that those ring can only support 1 direction swiping, either horizontal or vertical, hence the reason we go with IMU-based ring.
This demo does not use #ComputerVision for input gesture recognition.
#HumanCenteredDesigns #CHI #HCI #HID #Input
#AR #VR #MR
#ExtendedReality #VirtualReality #Metaverse
#UX #UI
3/5/2023
Phuoc Trinh