My Week With Project Tango

A few weeks back I got into Google’s exclusive Project Tango developers program. I’ve had a Tango tablet for about a week and have been experimenting with the available apps and Unity3D SDK.

Project Tango uses Movidius’ Myriad 1 Vision Processor chip (or “VPU”), paired with a depth camera not too unlike the original Kinect for the XBOX 360. Except instead of being a giant hideous block, it’s small enough to stick in a phone or tablet.

I’m excited about Tango because it’s an important step in solving many of the problems I have with current Augmented Reality technology. What issues can Tango solve?

POSITIONAL TRACKING

First, the Tango tablet has the ability to determine the tablet’s pose. Sure, pretty much every mobile device out there can detect its precise orientation by fusing together compass and gyro information. But by using the Tango’s array of sensors, the Myriad 1 processor can detect position and translation. You can walk around with the tablet and it knows how far and where you’ve moved. This makes SLAM algorithms much easier to develop and more precise than strictly optical solutions.

Also, another problem with AR as it exists now is that there’s no way to know whether you or the image target moved. Rendering-wise, there’s no difference. But, this poses a problem with game physics. If you smash your head (while wearing AR glasses) into a virtual box, the box should go flying. If the box is thrown at you, it should bounce off your head–big distinction!

Pose and position tracking has the potential to factor out the user’s movement and determine the motion of both the observer and the objects that are being tracked. This can then be fed into a game engine’s physics system to get accurate physics interactions between the observer and virtual objects.

OCCLUDING VIRTUAL CHARACTERS WITH THE REAL WORLD

Anyway, that’s kind of an esoteric problem. The biggest issue with AR is most solutions can only overlay graphics on top of a scene. As you can see in my Ether Drift project, the characters appear on top of specially designed trading cards. However, wave your hand in front of the characters, and they will still draw on top of everything.

Ether Drift uses Vuforia to superimpose virtual characters on top of trading cards.

Ether Drift uses Vuforia to superimpose virtual characters on top of trading cards.

With Tango, it is possible to reconstruct the 3D geometry of your surroundings using point cloud data received from the depth camera. Matterport already has an impressive demo of this running on the Tango. It allows the user to scan an area with the tablet (very slowly) and it will build a textured mesh out of what it sees. When meshing is turned off the tablet can detect precisely where it is in the saved environment mesh.

This geometry can possibly be used in Unity3D as a mesh collider which is also rendered to the depth buffer of the scene’s camera while displaying the tablet camera’s video feed. This means superimposed augmented reality characters can accurately collide with the static environment, as well as be occluded by real world objects. Characters can now not only appear on top of your table, but behind it–obscured by a chair leg.

ENVIRONMENTAL LIGHTING

Finally, this solves the challenge of how to properly light AR objects. Most AR apps assume there’s a light source on the ceiling and place a directional light pointing down. With a mesh built from local point cloud data, you can generate a panoramic render of where the observer is standing in the real world. This image can be used as a cube map for Image-based lighting systems like Marmoset Skyshop. This produces accurate lighting on 3D objects which when combined with environmental occlusion makes this truly a next generation AR experience.

A QUICK TEST

The first thing I did with the Unity SDK is drop the Tango camera in a Camera Birds scene. One of the most common requests for Camera Birds was to be able to walk through the forest instead of just rotating in place. It took no programming at all for me to make this happen with Tango.

This technology still has a long way to go–it has to become faster and more precise. Luckily, Movidius has already produced the Myriad 2, which is reportedly 3-5X faster and 20X more power efficient than the chip currently in the Tango prototypes. Vision Processing technology is a supremely nerdy topic–after all it’s literally rocket science. But it has far reaching implications for wearable platforms.

A Weekend at Oculus Connect

I spent this past weekend at Oculus Connect and have just now had the time to process what I saw. For Oculus to go from a humble Kickstarter project a few years ago to a capacity filled conference rife with amazing demos and prototypes by countless developers is mind-boggling. I know I said VR in 2014 is like Mobile in 2002, but the pace of progress is staggering. The maturation path for VR is going to be MUCH quicker. Is it 2005 already?

...and all I got was this lousy t-shirt.

…and all I got was this lousy t-shirt.

As I stated before, Gear VR is the most important wearable platform in the universe. I’ve been developing Gear VR games for a while and am thoroughly convinced this wireless, lightweight platform will have far more reach than VR tethered to your desktop.

The GearVR demo area.

The GearVR demo area.

The apps on display were great, but I even saw a few Gear VR demos from random developers in the hotel hallways that blew away what were officially shown in Samsung’s display area. Developer interest for Gear VR is very high. Once it’s commercially available, a flood of content is soon upon us.

Despite the intense interest in the platform, I spoke to a few desktop and console developers who dismissed Gear VR as a distraction and are ignoring it–which I think is really short-sighted.

It’s true that there may be a division in audiences. Gear VR may be the larger, casual audience while apps built around Oculus’ astounding Crescent Bay platform could be for a highly monetizable market of core enthusiasts. Either route is smart business. Depending on how long you can hold out for customer traction, that is.

Oh, and Crescent Bay…was a revolution. There’s probably not much more to be said about it that hasn’t already–but the ridiculous momentum behind Oculus’ path from the DK1 to Crescent Bay makes me question the competition. Oculus has hired all of the smartest people I know and have billions of dollars to spend on VR R&D–which is their main business, not a side project. Will competitors like Sony really commit enough resources to compete with the relentless pace of Oculus’ progress?

Why Consoles Didn’t Die

Yeah I was wrong. But hey, so were a lot of people. The PS4 barrels ahead with the fastest selling console ever. Microsoft is making a lot of similar but highly qualified statements about the XBOX One which leads me to believe it’s lagging behind significantly. Still, the recent European price cut and upcoming tent pole releases may perk things up.

photo (39)

Regardless, most console doom predictions haven’t come true. This is because Microsoft, and primarily Sony, changed their business models in response to the looming threat from mobile and tablets. If consoles kept going the direction they were in 2008, we would see a totally different story.

What changed?

No more loss leaders.

Consoles historically launched as high-end hardware sold at a loss–but still quite expensive. This peaked last generation with the ‘aspirational’ PS3 debuting at nearly $600 in 2006. The idea behind this business model was that they’d make it up in software sales and eventually cost reduce the hardware.

This time, Sony took a page out of Nintendo’s book and built lower cost hardware that can at least be sold close to breakeven at launch. The downside being that the tech specs are somewhat mundane. Price sensitivity wins over performance.

Dropping the gates.

The tightly gated ecosystem that dominated consoles for decades would have been absolutely disastrous if left to stand. Sony has largely obliterated their gate and gone for a more authoritarian version of Apple’s curated model. Surely the most significant evidence of mobile’s influence on console to date. Microsoft has also adopted this posture with their ID program. The indie revolution is heavily influencing games, and allowing this movement to continue on consoles is a smart move. Especially when fewer and fewer studios can execute at a AAA level.

Users didn’t move.

A lot of analysts mistook stagnant console numbers for lagging demand. It turns out, there really was just nothing else to buy. Despite hype about core games on mobile–that transition has yet to happen. Most titles console players would recognize as ‘core’ games have utterly failed to gain traction on tablets. Core gamers want core games exclusively on console or desktop while reserving mobile for a completely different experience.

Eventually we’ll see a major disruption in how and where games are consumed. It’s going to take longer than one console generation to transform core gamer habits. It also may be too early to tell. After all, we’re only a few months into this generation.

The fact is, the AAA economy isn’t sustainable. Massive layoffs, even while Sony is basking in post-hardware launch success, shows not all is well with the AAA end of the spectrum.

From Bits to Atoms: Creating A Game In The Physical World

Some of you may recall last year’s post about 3D printing and my general disappointment with consumer-grade additive manufacturing technology. This was the start of my year-long quest to turn bits into atoms. Since that time there has been much progress in the technology and I’ve learned a lot about manufacturing. But first, a little about why I’m doing this, and my new project titled: Ether Drift.

Ether Drift AR App

A little over a year ago, I met a small team of developers who had a jaw-dropping trailer for a property they tried to get funded as a AAA console game. After failing to get the game off the ground it was mothballed until I accidentally saw their video one fateful afternoon.

With the incredible success of wargaming miniatures and miniature-based board game campaigns on Kickstarter, I thought one way to launch this awesome concept would be to turn the existing game assets into figurines. These toys would work with an augmented reality app that introduces the world and the characters as well as light gameplay elements. This would be a way to gauge interest in the property before going ahead with a full game production.

A lot of this was based on my erroneous assumption that I could just 3D print game models and ship them as toys. I really knew nothing about manufacturing. Vague memories of Ed Fries’ 3D printing service that made figurines out of World of Warcraft avatars guided my first steps.

3D printers are great prototyping tools. Still, printing the existing game model took over 20 hours and cost hundreds of dollars in materials and machine time. Plus, 3D prints are fragile and require a lot of hand-finishing to smooth out. When manufacturing in quantity, you need to go back to old-school molding.

You can 3D print just about any shape, but molding and casting has strict limitations. You have to minimize undercut by breaking the model up into smaller pieces that can be molded and assembled. The game model I printed out was way too complicated to be broken down into a manageable set of parts.

Most of these little bits on the back and underside would have to be individual molded parts to be re-assembled later--An expensive process!

Most of these little bits on the back and underside would have to be individual molded parts to be re-assembled later–An expensive process!

So I scrapped the idea of using an existing game property. Instead, I developed an entirely new production process. I now create new characters from scratch that are designed to be molded. This starts as a high detail 3D model that is printed out in parts that molds are made from. Then, I have that 3D model turned into something that can be textured and rigged for Unity3D. There are some sacrifices made in character design since the more pieces there are, the more expensive it is to manufacture. Same goes for the painting process–the more detailed the game texture is, the more costly it becomes to duplicate in paint on a plastic toy.

We're working on getting a simple paint job that matches the in-game texture.

We’re working on getting a simple paint job that matches the in-game texture.

So, what is Ether Drift? In short: it’s Skylanders for nerds. I love the concept of Skylanders–but, grown adult geeks like toys too. The first version of this project features a limited set of figures and an augmented reality companion app.

The app uses augmented reality trading cards packed with each figure to display your toy in real-time 3D as well as allowing you to use your characters with a simple card battle game. I’m using Qualcomm’s Vuforia for this feature–the gold standard in AR.

The app lets you add characters to your collection via a unique code on the card. These characters will be available in the eventual Ether Drift game, as well as others. I’ve secured a deal to have these characters available in at least one other game.

If you are building a new IP today, it’s extremely important to think about your physical goods strategy. Smart indies have already figured this out. The workflow I created for physical to digital can be applied to any IP, but planning it in advance can make the process much simpler.

In essence, I’m financing the development of a new IP by selling individual assets as toys while it is being built. For me, it’s also a throwback to the days before everything was licensed from movies or comic books and toy store shelves were stocked with all kinds of crazy stuff. Will it work? We’ll see next month! I am planning a Kickstarter for the first series in mid-March. Stay Tuned to the Ether Drift site, Facebook page, or Twitter account. Selling atoms instead of bits is totally new ground for me. I’m open to all feedback on the project, as well as people who want to collaborate.

How to survive the mobile gaming apocalypse

I was listening to the latest Walled Garden podcast and towards the end they stopped just short of stating what many developers I talk to have been saying–mobile gaming is dead.

Ok, not actually dead. After all, mobile gaming revenue is higher than it’s ever been, and mobile consumption of everything is eating the planet. However, mobile gaming is completely dead as a business model for independent developers and undercapitalized startups.

IAP has become so dominant that there’s really only one somewhat reproducible way to make money in the AppStore: make a hamster wheel f2p game in a handful of established genres and spend tens of thousands of dollars a day on user acquisition to drive traffic to it. Despite many bold experiments, the charts increasingly bear this out.

Republique

This means that some companies with top charting mobile games aren’t actually making a profit as UA costs can eat up most of the revenue. Surely this will produce a shakeout and consolidation in 2014. This is similar to what happened to Facebook games circa 2010 causing a mass exodus to mobile.

Now that mobile is dead, where should you escape to? There are several options.

PC

The PC, and more specifically Steam, remains the platform of choice for those who actually want to charge money for content. There’s a large market for premium games and Steam has loosened their gate with the advent of Greenlight. Some prominent developers have been abandoning mobile for PC with their new projects. Despite PC sales declining in the face of tablets, it makes sense. This is where the paying customers are.

Consoles

A lot has been written here about the impending demise of consoles, but Sony and Microsoft managed to change up their business model and product strategy enough to have early success with both the PS4 and XBOX One. One of the big changes has been the thawing of the gated ecosystem and allowing independent developers to self-publish. Oh yeah, and on the Wii U also.

Next generation console owners are starved for content. There will be many independent successes over the next few years before the channel becomes completely saturated.

VR

On one hand VR is merely a peripheral for existing games, on the other it’s part of an entirely new category of wearable computing and an emerging platform. Oculus Rift is the clear leader with a huge round of investment and development kits widespread. However a glut of VR headsets is on the horizon.

Oculus is building an ecosystem out of their device, but VR content can be distributed through any PC gaming channel. Although, supporting every single headset may be a nightmare for developers–isn’t it time for some kind of standard VR API?

Board games

Board games are a cottage industry yet a hot category on Kickstarter. As an example, Sandy Peteresen’s Cuthulu World Combat iOS game Kickstarter failed miserably, but when re-pitched as a board game, it blew past its funding goal. Going from digital to physical presents a lot of new challenges for developers, but does have a dedicated fan base of paying customers. Plus, you can’t pirate a board game!

Facebook / Web

Facebook games ‘died’ in 2010, but are ironically becoming an increasingly common alternative platform for mobile developers. Especially if you have a working web client already, why not put it on Facebook? The problem is the audience is decidedly non-hardcore. Facebook games can still make some money, but for a very specific audience. However, for hardcore games, the open web still remains a viable place to find an audience of paying gamers. Kongregate proves this.

What needs to change in mobile?

The supremacy of f2p and the very few options for user acquisition make the momentum towards free and the companies with enough money to compete in the mobile UA wars insurmountable. Apple could make some changes to the App Store to help support premium games and other alternative business models, however there really isn’t any incentive to do so–Either way, Apple sells phones. It’s difficult to foresee anything but the continuing dominance of f2p and mega-publishers on mobile in 2014. If you have a ton of cash and resources, solving this problem is hard, and thus very lucrative. For the rest of us, plan your strategy accordingly.

How to go to a Conference

With my GDC Taipei Summit talk, I wrapped up a year of touring the world public speaking. It started with my GDC Online talk on iOS and Android development with Unity3D and concluded with a few trips to Asian GDCs with an expanded version of that talk. In between I did a few other presentations, including co-hosting the awards at Casual Connect SF.

Live from GDC Taipei Summit!

I figured I’d share my tips on how to effectively go to a conference as an entrepreneur.

Conferences are about networking, not information

You rarely learn anything at a conference. This is because the vast majority of speakers are shamelessly promoting themselves or their companies. Most lectures are thinly veiled advertisements. Also, absolutely never go to a panel. You’ll hear better conversations in the hotel bar afterwards. Never listen to sober panelists.

Conferences are about who you meet and the connections you can make. The best way to do this is to make yourself highly visible. Which leads to my second point.

Only attend conferences you are speaking at

If you have fears about public speaking, you’re just going to have to get over it. If other people are merely promoting themselves or their companies, why shouldn’t you? Naturally, an effective speaker shouldn’t shill–and if you are obviously self-pimping, you’ll never get invited back to speak. It’s a delicate balance.

Speaking is a great networking opportunity because you also get access to exclusive events such as speaker dinners that allow you to network with others who may be able to unlock some opportunities for you.

Target your audience

Speaking is all about attracting the right audience with the types of people you want to network with. You need to cater your talk to these people. For instance, this year my talk was highly technical. Technical talks are useless for business development purposes as the only people who attend are engineers–the so-called ‘losers’ of the proverbial Gervais pyramid.

If you’re trying to generate revenue for your business, speaking about the “why” and not the “how” of your subject is a better idea. Discussing why something should be done may attract more decision makers and less actual productive people.

On the other hand, technical talks are great for recruiting talent. I know many technical founders that give in-depth engineering presentations so they can recruit candidates from the audience afterwards. But please–don’t end with a “we’re hiring” slide.

Don’t have meetings

Don’t have meetings at a conference. Most gatekeepers and decision makers are merely going to conferences as a paid vacation. (See one of my earliest posts) They’re hovering within a gradient of two states: drunk or hungover–and if not, are intensely focused on getting laid before the end of the show and thus won’t remember anything you discussed. Try to get the commitment to meet them at their workplace after the conference–when they are presumably sober and have no choice but to listen to you.

Those are just some quick tips from years of attending conferences. Have an objective when you go. When someone asks you what you are doing, have a story to tell. If you just aimlessly wander trying to make serendipity happen, it never will. Self promote, and have a mission.

The Future of Physical Retail for Games

The fate of retail is grim. Best Buy is shuttering stores and showrooming threatens to destroy whatever’s left of the physical goods ecosystem. With digital downloads from Steam, the App Store, and the rise of web and social gaming it seems physical retail for games is going the way of the Compact Disc.

Yet, a strange thing is happening to music–At least in hip-hop circles (where I almost exclusively dwell).

Hip hop is the nerdiest music on earth. Super rap fans will wax nostalgic about who used what obscure sample first, the never ending saga of beef, and even the jacket color a rare 12 inch came in. Many of hip-hop’s biggest fans are obsessive collectors. To the untrained eye, there may be little difference between an episode of Hoarders and a look inside Just Blaze’s apartment.

Regardless, are 12 inch records a relic to be lost in the era of digital downloads as the entire economy shifts from atoms to bits?

The record label, Get On Down, is a product of the deluxe hip-hop album reissue trend from outfits like Traffic Entertainment started in the mid-2000s. These are completely remastered albums with hard to find remixes and brand new liner notes. Since most consumers download, stream, or steal their music, Get on Down needed to create another reason to purchase the product. As a result, they came up with a new formula: including exclusive collectibles with the album.

How about a Biz Markie jigsaw puzzle? An MF Doom Lunchbox? Or this incredible wooden case containing a perfect replica of one of hip-hop’s most prized items, the legendary Purple Tape? It’s like the Ark of the Covenant, except your face won’t melt off when it’s opened. Well, unless you’ve never heard Incarcerated Scarfaces. ‘90s babies prepare.

Get On Down’s production of limited edition collectibles dials right into the obsessive collector gene in hip hop fans. Other labels have similar strategies, such as Chopped Herring’s super limited vinyl pressings from artists both old and new. Fans are spending $40 for a new single and hundreds of dollars on eBay after the limited run expires.

Games can learn a lot from this new era of hip-hop music. The physical disc a game resides on is near meaningless in 2013. Still, that doesn’t mean gamers don’t want physical goods. Activison’s huge success with Skylanders proves that. Disney’s massive bet on Infinity is more evidence. For older gamers, look at the many successful Kickstarter campaigns featuring detailed plastic miniatures. Most people can’t be actually playing the games associated with these figures. They are collecting cool plastic belonging to an artist they like. Sound familiar?

There may be a backlash against digital games and virtual goods leading to an era of limited edition physical goods aimed at a small, but obsessive audience. Fans still want to own a piece of an IP they can touch.

In the past, collector’s editions have come with such ridiculous props as night vision goggles and console skins. It’s time to get serious about physical goods as an integral part of a game’s IP. High quality miniatures is a good place to start. Looking at Get On Down’s product line, you can see how they fit the physical product to match the artist. Each game deserves it’s own carefully crafted set of physical goods.

In the future, physical retail may only be an expensive collector’s edition without a disc–Just an ornate box containing high quality figures and a download code.