SoftKinetic explained their approach to improved AR as being three-pronged: Hardware, imaging, and software. First, they designed and produced a Time of Flight (ToF) device to accurately sense depth. Like most ToF devices, theirs uses a laser to perform the depth sensing at the industry standard 320x240 resolution. The second piece of SoftKinetic's approach is a 720p RGB color imager used to recognize gestures. These two hardware components (along with the logic needed to control them) make up the company's new gesture-recognizing input camera.
SoftKinetic had two different versions of the camera on-site (shown below). They're fairly small -- to the point that one laptop had a version embedded into its top bezel. That means no lugging an extra peripheral around.
The final piece of SoftKinetic’s approach is their software. They refer to it as iisu, or the Interface Is You.
What makes this different than just strapping on an Oculus Rift is that -- with the addition of the camera to the front -- you can now use your hands to manipulate the 3D environment in real-time. Using your hands as the input device is not new, but the level at which their demonstration performed was commendable. For this demo, there were no thimbles to put on our fingertips, there were no speed restrictions on our movements, and the one restriction they did give us we learned during the demo was no longer valid. (SoftKinetic's rep said we couldn’t manipulate two objects simultaneously—one with each hand—but we did it anyway and it turns out it worked!).
We haven’t had a chance to complete extensive talks with SoftKinetics so we can’t tell you exactly, technically, what makes their tech so smooth in its functionality, but as we learn more we promise to keep you updated.
The possibilities for this technology are many-fold. In the demo we experienced, which was created in Unity, we were given a block spawn point where we could grab blocks and put them anywhere in a 3D space. When we began stacking blocks and building small structures, the obvious came to mind: Minecraft. Building large, complex structures in Minecraft-like games seems like it'd be infinitely more intuitive and painless using such a natural input. SoftKinetic's partnership with Intel has a collection of more already-functional ideas over here. If you’re interested in creating your own 360* environment where your hands are the interface, it’s surprisingly accessible to developers. Here are the tools:
- Oculus Rift - https://www.oculusvr.com/order/ - $300.
- Creative SENZ3D camera - ~$180.
- Bracket to connect the two pieces of hardware available on MakerBot’s Thingiverse - $Free.
- iisu middleware - 90 day free trial (Linux & Android available upon request).
- Unity 3D.
We look forward to further exploring this technology with SoftKinetic! Keep your eyes out for future full-length pieces on this technology.
- Patrick "MoCalcium" Stone.