How To Support Gear VR and Google Cardboard In One Unity3D Project

Google Cardboard is a huge success. Cardboard’s userbase currently dwarfs that of Gear VR. Users, investors, and collaborators who don’t have access to Gear VR often ask for Cardboard versions of my games. As part of planning what to do next with Caldera Defense, I decided to create a workflow to port between Gear VR and Cardboard.

Always keep a Cardboard on me at ALL TIMES!

I used my VR Jam entry, Duck Pond VR, as a test bed for my Unity3D SDK switching scripts. It’s much easier to do this on a new project. Here’s how I did it:

Unity 4 vs. Unity 5

Google Cardboard supports Unity 4 and Unity 5. Although Oculus’ mobile SDK will technically work on Unity 5, you can’t ship with it because bugs in the current version of Unity 5 cause memory leaks and other issues on the Gear VR hardware. Unity is working on a fix but I haven’t heard any ETA on Gear VR support in Unity 5.

This is a bummer since the Cardboard SDK for Unity 5 supports skyboxes and other features in addition to the improvements Unity 5 has over 4. Unfortunately you’re stuck with Unity 4 when making a cross-platform Gear VR and Cardboard app.

Dealing With Cardboard’s Lack of Input

Although Gear VR’s simplistic touch controls are a challenge to develop for, the vast majority of Cardboards have no controls at all! Yes, Google Cardboard includes a clever magnetic trigger for a single input event. Yet, the sad fact is most Android devices don’t have the necessary dock connector to use this.

You have a few other control options that are universal to all Android devices: the microphone and Bluetooth controllers. By keeping the microphone open, you can use loud sounds (such as a shout) to trigger an action. You can probably use something like the Pitch Detector plug-in for this. Or, if your cardboard has a head strap for hands-free operation, you can use a Bluetooth gamepad for controls.

Because of this general lack of input, many Cardboard apps use what I call “stare buttons” for GUIs. These are buttons that trigger if you look at them long enough. I’ve implemented my own version. The prefab is here, the code is here. It even hooks into the new Unity UI event system so you can use it with my Oculus world space cursor code.

Gear VR apps must be redesigned to fit within Cardboard’s constraints. Whether it’s for limited controls or the performance constraints of low end devices. Most of my Cardboard ports are slimmed down Gear VR experiences. In the case of Caldera Defense, I’m designing a simplified auto-firing survival mode for the Cardboard port. I’ll merge this mode back into the Gear VR version as an extra game mode in the next update.

Swapping SDKs

This is surprisingly easy. You can install the Cardboard and Gear VR SDKs in a single Unity project with almost no problems. The only conflict is they both overwrite the Android manifest in the plugin folder. I wrote an SDK swapper that lets you switch between the Google Cardboard and Oculus manifests before you do a build. You can get it here. This editor script has you pick where each manifest file is for Cardboard and Gear VR and will simply copy over the appropriate file to the plugin folder. Of course for iOS Cardboard apps this isn’t an issue.

Supporting Both Prefabs

Both Oculus and Cardboard have their own prefabs that represent the player’s head and eye cameras. In Caldera Defense, I originally attached a bunch of game objects to the player’s head to use for traces, GUI positioning, HUDs, and other things that need to use the player’s head position and orientation. In order for these to work on both Cardboard and Oculus’ prefabs, I placed all objects attached to the head on another prefab which is attached to the Cardboard or Oculus’ head model at runtime.

Wrapping Both APIs

Not only do both SDK’s have similar prefabs for the head model, they also have similar APIs. In both Cardboard and Oculus versions, I need to refer to the eye and head positions for various operations. To do this, I created a simple class that detects which prefab is present in the scene, and grabs the respective class to wrap the eye position reference around. The script is in the prefab’s package.


For the final step, I made separate Cardboard versions of all my relevant Gear VR scenes which include the Cardboard prefabs and modified gameplay and interfaces. If no actual Oculus SDK code is in any of the classes used in the Cardboard version, the Oculus SDK should be stripped out of that build and you’ll have no problem running on Cardboard. This probably means I really need to make an Oculus and Cardboard specific versions of that CameraBody script.

The upcoming Unity 5.1 includes native Oculus support which may make this process a bit more complicated. Until then, these steps are the best way I can find to support both Cardboard and Gear VR in one project. I’m a big fan of mobile VR, and I think it’s necessary for any developer at this early stage of the market to get content out to as many users as possible.

Samsung Gear VR Development Challenges with Unity3D

As you may know, I’m a huge fan of Oculus and Samsung’s Gear VR headset. The reason isn’t about the opportunity Gear VR presents today. It’s about the future of wearables–specifically of self-contained wearable devices. In this category, Gear VR is really the first of its kind. The lessons you learn developing for Gear VR will carry over into the bright future of compact, self-contained, wearable displays and platforms. Many of which we’ve already started to see.

The Gear VR in the flesh (plastic).

The Gear VR in the flesh (plastic).

Gear VR development can be a challenge. Rendering two cameras and a distortion mesh on a mobile device at a rock solid 60fps requires a lot of optimization and development discipline. Now that Oculus’ mobile SDK is public and having worked on a few launch titles (including my own original title recently covered in Vice), I figured I’d share some Unity3D development challenges I’ve dealt with.


The biggest challenge with making VR performant on a mobile devices is throttling due to heat produced by the chipset. Use too much power and the entire device will slow itself down to cool off and avoid damaging the hardware. Although the Note 4 approaches the XBOX 360 in performance characteristics, you only have a fraction of its power available. This is because the phone must take power and heat considerations in mind when keeping the CPU and GPU running at full speed.

With the Gear VR SDK you can independently tell the device how fast the GPU and CPU should run. This prevents you from eating up battery when you don’t need the extra cycles, as well as tune your game for performance at lower clock speeds. Still, you have to be aware of what types of things eat up GPU cycles or consume GPU resources. Ultimately, you must choose which to allocate more power for.


The obvious optimization is lowering graphical detail. Keep your polycount under 50k triangles. Avoid as much per pixel and per vertex processing as possible. Since you have tons of RAM but relatively little GPU power available–opt for more texture detail over geometry. This includes using lightmaps instead of dynamic lighting. Of course, restrict your usage of alpha channel to a minimum–preferably for quick particle effects, not for things that stay on the screen for a long period of time.

Effects you take for granted on modern mobile platforms, like skyboxes and fog, should be avoided on Gear VR. Find alternatives or design an art style that doesn’t need them. A lot of these restrictions can be made up for with texture detail.

A lot of standard optimizations apply here–for instance, use texture atlasing and batching to reduce draw calls. The target is under 100 draw calls, which is achievable if you plan your assets correctly. Naturally, there are plenty of resources in the Asset Store to get you there. Check out Pro Draw Call Optimizer for a good texture atlasing tool.


There are less obvious optimizations you might not be familiar with until you’ve gone to extreme lengths to optimize a Gear VR application. This includes removing as many Update methods as possible. Most update code spent waiting for stuff to happen (like an AI that waits 5 seconds to pick a new target) can be changed to a coroutine that is scheduled to happen in the future. Converting Update loops to coroutines will take the burden of waiting off the CPU. Even empty Update functions can drain the CPU–death by a thousand cuts. Go through your code base and remove all unnecessary Update methods.

As in any mobile game, you should be pooling prefabs. I use Path-o-Logical’s PoolManager, however it’s not too hard to write your own. Either way, by recycling pre-created instances of prefabs, you save memory and reduce hiccups due to instantiation.


There’s nothing really new here to most mobile developers, but Gear VR is definitely one of the bigger optimization challenges I’ve had in recent years. The fun part about it is we’re kind of at the level of Dreamcast-era poly counts and effects but using modern tools to create content. It’s better than the good old days!

It’s wise to build for the ground up for Gear VR than to port existing applications. This is because making a VR experience that is immersive and performant with these parameters requires all disciplines (programming, art, and design) to build around these restrictions from the start of the project.

Oculus Rift World Space Cursors for World Space Canvases in Unity 4.6

Unity 4.6 is here! (Well, in public beta form). Finally–the GUI that I’ve waited YEARS for is in my hands. Just in time, too. I’ve just started building the GUI for my latest Oculus Rift project.

The new GUI in action.

The new GUI in action from Unity’s own demo.

One of the trickiest things to do in VR is a GUI. It seems easy at first but many lessons learned from decades of designing for the web, apps, and general 2D interfaces have to be totally reinvented. Given we don’t know what the standard controls may be for the final kit, many VR interfaces at least partially use your head as a mouse. This usually means having a 3D cursor floating around in world space which bumps into or traces through GUI objects.

Unity 4.6’s GUI features the World Space Canvas–which helps greatly. You can design beautiful, fluid 2D interfaces that exist on a plane in the game world making it much more comfortable to view in VR. However, by default Unity’s new GUI assumes you’re using a mouse, keyboard, or gamepad as an input device. How do you get this GUI to work with your own custom world-space VR cursor?

The answer is the use of Input Modules. However, in the current beta these are mostly undocumented. Luckily, Stramit at Unity has put up the source to many of the new GUI components as part of Unity’s announced open source policy. Using this code, I managed to write a short VRInputModule class that uses the result of a trace from my world space VR cursor and feeds it into the GUI. The code is here. Add this behavior to the EventSystem object where the default ones are.

In my current project, I have a 3D crosshair object that floats around the world, following the user’s view direction. The code that manages this object performs a trace, seeing if it hit anything in the UI layer. I added box colliders to the buttons in my World Space Canvas. Whenever the cursor trace hits one of these objects, I call SetTargetObject in the VRInputModule and pass it the object the trace hit. VRInputModule does the rest.

Note that the Process function polls my own input code to see if a select button has been hit–and if so, it executes the Submit action on that Button. I haven’t hooked up any event callbacks to my Buttons yet–but visually it’s responding to events (highlighting, clicking etc.)

It’s quick and dirty, but this should give you a good start in building VR interfaces using Unity’s new GUI.

The Next Problems to Solve in Augmented Reality

I’m totally amped up about Project Tango. After having worked with augmented reality for a few years, most of the problems I’ve seen with current platforms could be solved with a miniaturized depth-sensing Kinect-style sensor. The Myriad 1 is a revolutionary chip that will dramatically change the quality of experience you get from augmented reality applications–both on mobile devices and wearables.

There’s a few other issues in AR I’d like to see addressed. Perhaps they are in research papers, but I haven’t seen anything real yet. Maybe they require some custom hardware as well.

Real-world lighting simulation.

One of the reasons virtual objects in augmented reality look fake is because AR APIs can’t simulate the real-world lighting environment in a 3D engine. For most applications, you place a directional light pointing down to and turn up the ambient for a vague approximation of overhead lighting. This is assuming the orientation of the object you’re tracking is upright, of course.

Camera Birds AR mode using an overhead directional light.

Camera Birds AR mode using an overhead directional light.

What I’d really like to use is Image Based Lighting. Image based Lighting is a computationally efficient way to simulate environmental lighting without filling a scene up with dynamic lights. It uses a combination of cube maps built from HDR photos with custom shaders to produce great results. A good example of this is the Marmoset Skyshop plug-in for Unity3D.

Perhaps with a combination of sensors and 360 cameras you can build HDR cubemaps out of the viewer’s local environment in real-time to match environmental lighting. Using these with Image Based Lighting will be a far more accurate lighting model than what’s currently available. Maybe building rudimentary cubemaps out of the video feed is a decent half-measure.

Which object is moving?

In a 3D engine, virtual objects drawn on top of image targets are rendered with two types of cameras. Ether the camera is moving around the object, or the object is moving around the camera. In real life, the ‘camera’ is your eye–so the it should move if you move your head. If you move an image target, that is effectively moving the virtual object.

Current AR APIs have no way of knowing whether the camera or the object is moving. With Qualcomm’s Vuforia, you can either tell it to always move the camera around the object, or to move the objects around the camera. This can cause problems with lighting and physics.

For instance, on one project I was asked to make liquid pour out of a virtual glass when you tilt the image target it rest upon. To do this I had to force Vuforia to assume the image target was moving–so then the image target tilted, so would the 3D object in the game engine and liquid would pour. Only problem is, this would also happen if I had moved the phone as well. Vuforia can’t tell what’s actually moving.

There needs to be a way to accurately track the ‘camera’ movement of either the wearable or mobile device so that in the 3D scene the camera and objects can be positioned accurately. This will allow for lighting to be realistically applied and for moving trackable objects to behave properly in a 3D engine. Especially with motion tracking advances such as the M7 chip, I suspect there are some good algorithmic solutions to factoring out the movement of the object and the observer to solve this problem.

Anyway, these are the kind of problems you begin to think about when staring at augmented reality simulations for years. Once you get over the initial appeal of AR’s gimmick, the practical implications of the technology poses many questions. I’ve applied for my Project Tango devkit and really hope I get my hands on one soon!

Ludum Dare: Ten Seconds of Thrust

This past weekend I participated in Ludum Dare 48, a contest where you make a game by yourself in 48 hours. The theme is revealed at the start of the contest–Iron Chef style. All code, graphics, and sounds have to be made from scratch. Voting began on Sunday night and will extend for a few weeks. I’m not even sure what you win, but that’s not the point. It’s an awesome experience in GETTING IT DONE.

Ten Seconds of Thrust!

My entry is the Lunar Lander-esque Ten Seconds of Thrust. (Please rate it!) Attempt to land at the bottom of increasingly difficult randomly generated space caverns with only ten seconds of thruster time. It’s crude, ugly, and buggy–especially on Windows where it doesn’t seem to detect landing. I didn’t have time to fix this bug as I only discovered it in the last half hour, but it does seem like a strange Unity Web Player bug since it works fine on OSX browsers. (PROTIP: Make sure you have a few friends around during the weekend to test your game!)

One of the best things about the contest is watching games evolve quickly through Twitter, Vine, Facebook, and Instagram posts. I put up a few videos in progress over the weekend.

I used a lot of the tools mentioned in my rapid prototyping posts, including a new tool I found called Sprite Gen which creates randomly generated animated character and tile sprites in tiny 12×12 blocks. Naturally, the game was developed in Unity along with 2DToolkit and HOTween for plug-ins.

I’d like to fix the landing bug as it makes the game useless on Windows, but the rules are somewhat unclear on bug-fixes that don’t add any content. This game was actually based on an idea for a Lunar Lander roguelike I was developing earlier this year. The LD48 version is highly simplified and way more fun. I abandoned my prototype in disgust back in February. This quick and dirty version is much better–I might run with it and make a full game.

Big Data Bootstrapping Beware

I suppose this a dumb observation, but the one thing I learned in building ZenThousand is that bootstrapping a big data startup can be expensive. Obviously, it’s due to all of that data you have to deal with before having a single user.

First, there’s the problem of collecting the data. In the case of ZenThousand, I am looking for social network profiles of programmers. Although sites like Github, LinkedIn, and others have collected a treasure trove of personal details on engineers, it’s not like they are just going to let you walk in and take it!

In LinkedIn’s case, over half of their revenue is from their recruiting features. Essentially, the information they have on you is worth nearly a billion dollars. Which is why use of their people search API has strict licensing restrictions. Github and other social sites don’t let you do simple searches for users either–you have to use what tools they give you in their API to sniff info out.

Collecting your own data can be very expensive. Data intensive services such as mapping require massive effort. This is why companies sitting on large datasets are so valuable. People scoff at Foursquare’s valuation, but while the app might not have great user numbers the location database they’ve built is of immense value.

Secondly, there’s the cost to store and process all of this data. With most startups, the amount of data you store is directly proportional to the amount of users you have. Scaling issues become a so-called “good problem to have” as it usually means your app has a lot of traction. If these are paying users, even better–Your data costs are totally covered.

With a big data startup, you have massive amounts of data to store and process with no users. This gets costly really fast. In my case, Google App Engine service fees quickly became prohibitively expensive. My future strategy involves moving off of GAE and on to either Google Compute Engine or a physical box. I know of at least one big data startup that migrated out of the cloud to a colocation facility for both cost and performance reasons.

This doesn’t mean big data isn’t possible without a large investment. It’s just that two of the first big problems you need to solve are how to cost-effectively collect and analyze lots of data before you have any revenue.

Extremely Rapid Prototyping Tools Part 2

It’s been awhile since my last post on extremely rapid prototyping tools. So I figured I’d list a few gems I’ve discovered along the way. A lot of times having the right tools to create placeholder content, find clip art, or other resources can get you over the hump when trying to create a quick prototype. Here’s some tools that can help.

A Placeholder!

When laying out HTML/CSS, or otherwise mocking up interfaces, sometimes you need placeholder icons, pictures, and other bitmaps. I discovered this incredibly simple to use and absolutely ingenious site awhile back,, which does just that. You simply pass the dimensions of the bitmap to the URL like×150 and it will create a bitmap of that size. Why didn’t I think of this?

Need temp music for your game but don’t want to find a composer or pay for library tracks? Easy, use autotracker. This free python script will create surprisingly good randomly generated chiptune music in IT tracker format. This is a very compact format that can be used directly in engines such as Unity or converted to MP3 or WAV files. It even randomly generates a name for each composition.

sfxr / Cfxr / Bfxr

Need sounds for your prototype? These open source tools will create randomly generated 8-bit style sound effects using tunable parameters and sound types (power up, damage, jump, etc.) I find the quality of the flash version, Bfxr, to be the best. Especially if you are doing a retro 8-bit style game, as so many neckbeardy hipsters are, these sounds are good enough for production use too. I’ve been toying around with the idea of porting it to C# as a Unity3D editor extension.