My Week With HoloLens


Microsoft ships the HoloLens and Clicker accessory in the box

My HoloLens development kits finally arrived a week ago. I’ve spent a great deal of time using the device over the past week. I figured I’d post my impressions here.

This Really Works

When I first put my HoloLens on, I placed an application window floating above my kitchen table. Suddenly, I realized I hadn’t taken out the garbage. Still wearing the device, I ran downstairs to put something in the trash. I narrowly missed my neighbor–successfully avoiding an awkward conversation about what this giant contraption on my face does.

When I returned to my kitchen, the application was still there–hovering in space.

As I’ve stated before, Microsoft blew me away. HoloLens is an absolutely incredible leap from previous generation AR glasses (and MUCH cheaper, believe it or not). It also does everything Tango does but at a much higher level of performance and precision. Which means most applications built on Tango can be directly moved over to HoloLens.

HoloLens is fast and intuitive enough to attempt getting actual work done with it. Yet, a lot of my time spent is just trying to make silly videos like this.

It’s A Full Blown Windows Machine

HoloLens isn’t just a prototype headset–it’s a full featured desktop Windows PC on your face. Not only can you run “Windows Holographic” apps, but any Universal Windows App from the Windows Store. Instead of these applications running in a window on a monitor, they float around in space–positioned wherever you choose.

Although HoloLens really does need a taskbar of some kind. It’s way too easy to forget where Skype physically is because you launched it in the bathroom.

It also helps to connect a Bluetooth keyboard and mouse when running standard applications. Gestures can’t give you the input fidelity of a traditional mouse, and typing in the air is a chore.

HoloLens’ narrow FOV makes using a regular Windows app problematic–as the screen will get cut off and require you to move your head around to see most of it. Also, if you push a window far enough into the background so you can see the whole thing, you’ll notice HoloLens’ resolution is a little low to read small text. We’re going to need a next generation display for HoloLens to really be useful for everyday computing.

Microsoft Has Created A New Input Paradigm

HoloLens can seemingly only recognize two gestures: bloom and “air tap”. Blooming is cool–I feel like a person in a sci-fi movie making the Windows start menu appear in the air by tossing it up with a simple gesture.

The air tap can be unintuitive. Most people I let try the HoloLens poke at the icons by stabbing them with a finger. That’s not what the air tap is for. You still have to gaze at a target by moving your head and then perform the lever-like air tap gesture within the HoloLens camera’s view to select what the reticule is on.

HoloLens can track the motion of your finger and use it as input to move stuff around (such as application windows), but not detect collisions between it and virtual objects. It’s as if it can detect the amount your finger moves but not its precise location in 3D space.

Using apps while holding your hand out in front of the headset is tiring. This is why Microsoft includes the clicker. This is a simple Bluetooth button that when pressed triggers the air tap gesture. Disappointingly, the clicker isn’t trackable–so you can’t use it as a true finger replacement.

Microsoft has adapted Windows to the holographic model successfully. This is the first full blown window manager and gesture interface for augmented reality I’ve ever seen and it’s brilliant. After a few sessions with the device, most people I’ve let use it are launching apps and moving windows around the room like a pro.

This Thing Is Heavy

Although the industrial design is cool in a retro ‘90s way, this thing is really uncomfortable to use for extended periods of time. Maybe I don’t have it strapped on correctly, but after a 20 minute Skype session I had to take the device off. I felt pain above the bridge of my nose. When I looked in the mirror, I saw what can only be described as ‘HoloHead’


The unfortunate symptom of “HoloHead”

The First Generation Apps Are Amazing

There already are great free apps in the Windows Store that show off the power of the HoloLens platform. Many made by Asobo Studio–a leader in Augmented Reality game development.

Young Conker

Young Conker is a great example of HoloLens as a games platform. The game is simple: after scanning your surroundings, play a familiar platform game over the floors, walls, tables and chairs as Conker runs, jumps and collects coins scattered about your room.

Conker will jump on top of your coffee table, run into walls, or be occluded by a chair as if he were walking behind it–well, depending on how accurate your scan is. The fact that this works as well as it does is amazing to me.


One of the first true game experiences I’ve ever played in augmented reality. You play the part of a futuristic detective, revisiting memories of crimes as their events are re-created holographically in your location. Characters sit on your furniture. You’ll hunt for pieces of evidence scattered about your room–even under tables. It really is an incredible experience, As with Conker, it requires some pre-scanning of your environment. However, applications apparently can share scans between each other as Fragments was able to re-use a scan of my office I previously made with another app.


When Skyping with a person not using HoloLens, you simply place their video on a wall in your surroundings. It’s almost like talking to someone on the bridge of the Enterprise, depending on how big you make the video window.

When Skyping with another HoloLens user, you can swap video feeds so either participant can see through the other’s first person view. While looking at someone else’s video feed as a floating window, you can sketch over it with drawing tools or even place pictures from your photos folder in the other person’s environment. 2D lines drawn over the video feed will form around the other user’s real-world in 3D–bending around corners, or sticking to the ceiling. 


As a consumer electronics device, HoloLens is clearly beta–maybe even alpha, but surprisingly slick. It needs more apps. With Wave 2 underway, developers are working on just that. In my case, I’m moving all of my Tango projects to HoloLens–so you’ll definitely be seeing cool stuff soon!

From Bits to Atoms: Creating A Game In The Physical World

Some of you may recall last year’s post about 3D printing and my general disappointment with consumer-grade additive manufacturing technology. This was the start of my year-long quest to turn bits into atoms. Since that time there has been much progress in the technology and I’ve learned a lot about manufacturing. But first, a little about why I’m doing this, and my new project titled: Ether Drift.

Ether Drift AR App

A little over a year ago, I met a small team of developers who had a jaw-dropping trailer for a property they tried to get funded as a AAA console game. After failing to get the game off the ground it was mothballed until I accidentally saw their video one fateful afternoon.

With the incredible success of wargaming miniatures and miniature-based board game campaigns on Kickstarter, I thought one way to launch this awesome concept would be to turn the existing game assets into figurines. These toys would work with an augmented reality app that introduces the world and the characters as well as light gameplay elements. This would be a way to gauge interest in the property before going ahead with a full game production.

A lot of this was based on my erroneous assumption that I could just 3D print game models and ship them as toys. I really knew nothing about manufacturing. Vague memories of Ed Fries’ 3D printing service that made figurines out of World of Warcraft avatars guided my first steps.

3D printers are great prototyping tools. Still, printing the existing game model took over 20 hours and cost hundreds of dollars in materials and machine time. Plus, 3D prints are fragile and require a lot of hand-finishing to smooth out. When manufacturing in quantity, you need to go back to old-school molding.

You can 3D print just about any shape, but molding and casting has strict limitations. You have to minimize undercut by breaking the model up into smaller pieces that can be molded and assembled. The game model I printed out was way too complicated to be broken down into a manageable set of parts.

Most of these little bits on the back and underside would have to be individual molded parts to be re-assembled later--An expensive process!

Most of these little bits on the back and underside would have to be individual molded parts to be re-assembled later–An expensive process!

So I scrapped the idea of using an existing game property. Instead, I developed an entirely new production process. I now create new characters from scratch that are designed to be molded. This starts as a high detail 3D model that is printed out in parts that molds are made from. Then, I have that 3D model turned into something that can be textured and rigged for Unity3D. There are some sacrifices made in character design since the more pieces there are, the more expensive it is to manufacture. Same goes for the painting process–the more detailed the game texture is, the more costly it becomes to duplicate in paint on a plastic toy.

We're working on getting a simple paint job that matches the in-game texture.

We’re working on getting a simple paint job that matches the in-game texture.

So, what is Ether Drift? In short: it’s Skylanders for nerds. I love the concept of Skylanders–but, grown adult geeks like toys too. The first version of this project features a limited set of figures and an augmented reality companion app.

The app uses augmented reality trading cards packed with each figure to display your toy in real-time 3D as well as allowing you to use your characters with a simple card battle game. I’m using Qualcomm’s Vuforia for this feature–the gold standard in AR.

The app lets you add characters to your collection via a unique code on the card. These characters will be available in the eventual Ether Drift game, as well as others. I’ve secured a deal to have these characters available in at least one other game.

If you are building a new IP today, it’s extremely important to think about your physical goods strategy. Smart indies have already figured this out. The workflow I created for physical to digital can be applied to any IP, but planning it in advance can make the process much simpler.

In essence, I’m financing the development of a new IP by selling individual assets as toys while it is being built. For me, it’s also a throwback to the days before everything was licensed from movies or comic books and toy store shelves were stocked with all kinds of crazy stuff. Will it work? We’ll see next month! I am planning a Kickstarter for the first series in mid-March. Stay Tuned to the Ether Drift site, Facebook page, or Twitter account. Selling atoms instead of bits is totally new ground for me. I’m open to all feedback on the project, as well as people who want to collaborate.