Hardware Guides stub

First Look at castAR Augmented Reality Gaming Glasses at PAX

Posted on August 31, 2015

CastAR, formerly Technical Illusions, recently got a big boost in the form of a 15-million dollar venture capital investment. The company plans to use that money to deliver on promises to their original Kickstarter backers and push the product into a complete state. GN was able to spend an hour with castAR CEO David Henkel-Wallace and cofounder Rick Johnson to see where things stand and where the company is going.

CastAR is a head-mounted, augmented reality technology that deploys a set of projectors and lenses to cast a 3-dimensional image to a reflective sheet. When we say that castAR is an HMD, we don’t mean in the “expected” sense – it’s not like the Rift or HTC’s impressive Vive, but is more akin to nVidia’s 3D Vision glasses in form factor. CastAR is billed as a solution for multiplayer and singleplayer AR gaming, to include traditional tabletop emulation (D&D, miniatures, Magic, Jenga) and new games.

The Demo

I was able to try out five different software demos. CastAR started me out on their seasoned Marble Madness-like game where I went head-to-head with their comms/media relations specialist, Christina Engel (she won... by a lot). This was used to demonstrate the familiar multiplayer aspect and provide an introduction to the 3D world.

After that, we moved into the classic tabletop game Battleship, where we used the hand-held controllers to target the opponent's grid in hopes of landing a strike. The point of this title was to demonstrate that things could be made visible or invisible on a per-user basis. No matter where I moved in the room, I could not see my opponent’s boats. Co-founder Rick Johnson added that this could be used to provide equal challenge to very differently skilled players in the same game at the same time, a sort of handicapping system.

castar-cast1

The third demo took us in to an infinite Jenga game where the Jenga tower was constantly rising out of the retroreflective surface. During the demo, Henkel-Wallace paused the game and asked me to look more closely at the blocks. As I moved closer, the level of detail in each block became more obvious as I could then see a clear wood grain pattern. The idea was to demonstrate that proximity to an object changes the object's LOD, as it would in reality.

castar-ship1

castar-cast2

After infinite Jenga, castAR brought me into a chess game that closely resembled the one seen in the original Star Wars. David pointed out that this demo was used to show the graphical quality achievable with the system, although I was still focused on the differing sensitivity of the controller when interacting with the objects.

The final demo was an isometric shooter that allowed me to revisit my G. I. Joe days. If you’ve ever desired to have a really cool model space station setup where you can interact with attacking robots and reactive doors, then you might want to further investigate castAR. The takeaway from this demo was that users can really explore the 3D space. At one point, I got the character stuck on a wall and, instead of battling with the controls, I simply moved to where I could see the character better. It was nice not to be in an absolutely fixed space.

User Experience

Many people who have tried solutions like the Oculus Rift or HTC/Valve Vive have expressed some level of adjustment or discomfort upon removal of the headset. After the approximate 25-minute demo I went through, I noted that there was a little bit of tension around my forehead and temples from where the unit was mounted. There was also a 3-5 second period of slight vision blur. This is something I also experience after watching 2-hour 3D films. All things considered, the discomfort was minimal and the adjustment back to normal sight was quick.

The Hardware

All of these demos were presented on the current hardware -- version 16 -- the same stuff that has been shipped to early developers. The 130 gram headset appears to be the most complex piece of gear in the setup. In its current state it is equipped with two 1280x720 micro projectors that are just above the lenses, one for each eye. In between the two projectors are two 120Hz, camera-like sensors. In addition to these sensors there is also a gyroscope and magnetometer. These three sensor systems together are used to interpolate user positioning relative to the retroreflective surface. The lenses themselves are just polarized material, similar to what is used in 3D movie glasses. There is also a 3.5mm audio jack built in to the frame so that a gaming audio headset can be connected. The input signal is still being delivered via an HDMI cable to an FPGA in the frame that does a bit of processing.

The handheld controller closely resembles a Wii remote. It uses the same camera-like sensors as the glasses to help the system understand what the user wants to do. The current 3D-printed prototype has the same trigger positioning as the Wiimote and incorporates the Wiimote nunchuk joystick, opposite the trigger. Below the joystick is a set of five buttons. Henkel-Wallace stated that the button configuration has not been finalized yet, and that the company is seeking more input on the controller. Their forum here is the place for discussion on the topic.

The final pieces of the system are the retroreflective surface, a 120Hz transmitter, and the IO controller.

castar-sheet

The retroreflective surface, which is like the reflective material you find on running shoes, can vary in size and, as far as we could tell, the size is only limited by the maximum projection distance of the headset. For instance, at one point during the demonstration, Johnson placed a piece of the material on a wall about 5-6 meters away and drew my attention to it. I was able to see a cloud floating in the sky in all of its 3D detail. The key to this material and why it works so well is that it creates directional reflection instead of specular reflection. This means that light coming from the projectors is directed almost entirely back at the eye instead of straying off in multiple directions.

The purpose of the 120Hz transmitter pad is to aid in location tracking of the headset. When we asked about the positioning of the microelectronics on the surface of the transmitter pad, we were told that the central transmitter was offset to determine orientation. In other words, if the central transmitter were perfectly centered, then the orientation of the pad would not be unique and would make it impossible to determine the exact location of the headset.

castar-io

The final piece, the IO contoller, is a device built as an input/output interface between the handheld controller, the headset, the HDMI from the PC, and the USB from the PC.

The hardware in its current state is less-than-beautiful. Recall though that Technical Illusions just got a 15-milliion dollar shot in the arm. This spurred them to officially change their name to CastAR, and to set a flexible deadline of 18 months for a more refined, off-the-shelf product delivered to customers and retailers. Henkel-Wallace shared his vision of a 75-gram headset where the FPGA is swapped out for a smaller, lighter piece of finalized silicon. He also stated that the frame currently has lots of extra room inside that was put there in case of need during development. Now that the project is near completion, that should also be streamlined and made lighter.

With user input, a final version of the handheld controller will be prototyped and approved. The final step is to remove all the wires from the user devices, making them completely wireless. Most importantly, though, the team emphasized that the process of opening the box, putting on the gear, and playing should be easy enough for kids to do. Their driving goal from the beginning of the project has always been that their product must be fun. If it were too difficult to set up, the fun factor would be significantly reduced.

The Software

In terms of software, cofounder Rick Johnson, who is a software engineer himself, said that the system should be fun for developers, too. Several of their demonstration applications were developed using Unity, and he assured us that development for the castAR is no more difficult than current game development.

The Possibilities

castar-2

In the current AR/VR market, there appear to be several choices soon to be available. This solution is definitely on the AR side of things, although there is an available clip-on device that turns them into more of a VR solution. If the other vendors' VR solutions appear too bulky and you are more interested in a lightweight alternative that may have less ergonomic impact over long gaming sessions, this could be ideal. Checking out their website, the funding is allowing the team of 9 to continue to grow, which should improve the product and its support.

After experiencing the demo, one of my first use case thoughts was, “can this be used to layout and play a D&D campaign?" Johnson's reply was a simple, "Yes." He then went on to state that, due to the nature of the software, it's the same as existing networked games. If you can do it with current titles then you can do it with castAR. The difference would be that you would be interacting with a 3D environment and doing so in real-time with your friends. We’re definitely intrigued and await the development over the next 18 months.

- Patrick Stone.