My Podcast Debut

Just a quick note, My friend Peter Katz had me as a guest on his Hollywood 2.0 podcast today. You can listen to the episode here.

On this episode, I’m rambling for a about a half hour about games, development strategies, the realities of the business, and a bunch of other stuff. Hey, I talk fast so it’s over quick.

By the way, I used the Samson Meteor Mic with Skype to record this. What a great sounding mic–and super cheap! If anyone else has a podcast they are desperate to get guests for, my Meteor Mic and I are ready.

Amazon Mechanical Turk Survey Strategies

I recently crafted a survey to collect data on the consumption habits of gamers. I wanted to identify different types of gamers and drill down further with future surveys targeted at specific categories of game players. To do this, I used a combination of SurveyMonkey and Amazon Mechanical Turk.

Making the Survey

The first step was creating the survey. For this, I paid for an account at SurveyMonkey. I find SurveyMonkey not only has the easiest survey creation tools but great features for analyzing the data. Alternatively, you can create surveys in Google Docs for free or inside Mechanical Turk itself using HTML.

There’s a science to creating a survey. I never took a class in demography, so I relied on some books discussing proper survey technique.

A few quick tips:

  • Always ask about what the person has done, not what they will do. For instance, instead of asking how much money they would spend on a car, ask what they spent on their last car. You will get a more accurate answer.

  • Make sure they are paying attention. Especially if your survey is long, you may be vulnerable to users clicking randomly just to get through it. This Scientific American article has a neat technique for checking if the user is taking an online survey seriously. Just ask ridiculous questions at random points. If the user answers something like “Have you ever eaten a dinosaur?” positively, throw his data out. He’s obviously not giving you valid answers.

    • Amazon Mechanical Turk

    Mechanical Turk is Amazon’s micro task site they launched in 2006. You can create tasks (called HITs) of any sort and pay workers to complete them. For instance, give someone a nickel to tell you whether a username is profane or not. In my case, I paid users 25 cents to fill out my survey.

    I used this blog as a guide on how to do it. Except instead of making the survey as a web form inside the HIT’s HTML code, I embedded my SurveyMonkey survey.

    Some tips:

  • Don’t be cheap. At first, I was trying to get away with paying 10 cents per survey. At that rate, I only got 5 or so filled out a day. When I upped the price to 25 cents, I got 30-50 a day. If you look at this confidence interval calculator, you can see it takes about 400 responses to have a decent degree of accuracy. So at 25 cents you can probably get yourself 400 responses in a week. Up your payment to a dollar and you’ll get 400 in a day.

  • Don’t get excited by your first day numbers. For workers, HITs are primarily sorted by when they were created. On the first day you create your HIT it will be at the top of the list. You’ll see a significant drop off in activity after the first day. Instead of collecting results all week it might pay off to break it up into 2 day HITs. (SurveyMonkey tracks IPs to prevent users from filling your survey out twice)

  • Give workers enough time. When designing a HIT you specify a duration. Even if your survey takes 5 minutes to complete, make the HIT last at least 30 minutes. When I shortened the window to 10 minutes I started getting complaints about it expiring before workers could complete it. Yes, you will get emails from dissatisfied workers–even over a 10 cent task.

  • Amazon’s demographics are surprisingly close for the US. I had almost an even ratio of male to female responders. Racial demographics were a little off, but not enough for me to correct them. This article provides some good insight on who is using Mechanical Turk.

    • Sign up as a Mechanical Turk worker yourself to see how it’s being used. You’ll see a lot of surveys as well as tasks like transcribing audio clips or even calling customer service phone lines and rating the operator’s performance.

    You can use MTurk surveys to collect all sorts of important data on your product or service before you even start writing code. See if you are solving a real problem for paying customers with your start-up. I’ve even heard of people surveying to test how likely users are to click on different app icon designs. The possibilities are endless and the information invaluable.

    2011 Year in Review

    Location isn’t dead. It’s just everywhere.

    When Facebook announced they were killing Places, it was kind of a shock. Several months later it makes total sense. Location isn’t a feature people use apps for. Location is now part of the social web’s fabric.

    Instead of a separate Facebook feature, you can now tag a location on to anything: posts, photos, and (yes!) check-ins. It seems like location apps such as Loopt, Brightkite, and SCVNGR have faded as they all ‘pivot’ to Groupon clones or whatever.

    Now location is merely an expected feature. I’m willing to bet most people don’t use Instagram primarily because of location. Still, tagging locations on photos is popular. Instagram is outpacing Foursquare for growth. Probably because it’s actually useful beyond announcing “I’m here!” to complete strangers.

    Peak Soccer Mom.

    Zynga is a monster of a company and congrats on their IPO. I remember seeing Mark Pincus at Cassie Phillips’ very first social gaming conference 5 or so years ago as a relative nobody. This was back when SGN had more funding.

    Today, SGN is nothing but a memory. Zynga managed to upend the entire game industry, plunging the old guard into panic and chaos while blazing a trail to a 7 billion dollar valuation. Bravo!

    A lot of analysts are disappointed with the IPO. And yeah, Zynga desperately needs new customers. For this reason the company seems like it’s not a growth stock.

    Is it a risky investment? Perhaps not. Zynga is the first public company whose performance you can watch in real-time. Instead of looking at guidance and waiting for quarterly reports, just go to AppData and watch their DAUs. If only there was a way to automatically sell a stock once their user retention dropped across the board…

    Unity 3.5

    The Unity 3.5 beta is now public. The GUI is nowhere to be found–my #1 problem with Unity. Still, it has file locking for SVN and P4V support, multi-select, and a bunch of other small fixes that I’m happy with. Oh, and the Flash exporter really works. Plus, there are some promising new GUI add-ons in the Asset Store.

    Console Disruption

    The 3DS has been able to pull itself back from the brink with quality software this holiday season. This effort may prove futile as tablets and mobile devices are still pounding the handheld market into submission.

    I’m big on the Sony Vita. I even paid extra to reserve the First Edition bundle early. I am fully aboard this kamikaze mission to stuff every conceivable feature in a massively powerful device directly aimed at a market that wants nothing to do with it. A glorious death on the battlefield! I want to see how this turns out–even though a brutal, bloody loss for the axis is seemingly inevitable. Operation DOWNFALL.

    Folks I know that got a Japanese Vita say it’s a glorious piece of gaming hardware. However, the 3DS outsold the Vita during launch week in Japan. Not a good sign.

    Action Bronson Has The Best Album of the Year

    Action Bronson’s Well Done is the best album of the year. The first rapper to rhyme about how to properly cook and serve puffin. Real talk.

    Mackerel Management

    The only time I’ve ever been fishing is one trip in my early teens to fish for mackerel. Apparently this is the perfect beginner fishing trip because mackerel are really easy to catch. I remember staring in amazement as dangling a shiny lure above the water caused hordes of fish to leap into the boat. You didn’t need to hook them! I went from a total novice to catching 20 pounds of fish in one afternoon. Unfortunately, that meant eating mackerel out of my mom’s freezer for two months that Summer.

    In the software world, it’s not uncommon to be dealing with management that behaves much like mackerel. Let’s say you’ve been charged with coming up with some grand plan for a new product or changes to an existing one. (Yes, I know this is very anti lean startup, but mackerel haven’t caught up with that yet) You might come up with a ‘road map’ having all sorts of features to be deployed upon some imaginary timeline.

    The thing is, if you are working for mackerel, you have to understand that each option or feature is a shiny lure. When presented with a number of options, mackerel will jump on every single one. They are all sparkling, attractive, and worth risking a slow, gasping death aboard the deck of a fishing vessel for.

    The good thing is, mackerel are easily distracted. When managing mackerel, your job is to keep them hopping on lures and thinking they are making real decisions while you enable the people who actually do stuff to build and launch the product the right way. I’ve seen products ship using this technique. Producers convince the project leader he is in charge, but secretly direct the team to do otherwise. Once the project ships, nobody can really complain. It’s the whole “ask forgiveness later than permission before” thing.

    Mackerel management requires two things. The first one is not caring about your employment status. After all, you are directly defying the mackerel and might get canned for such brazen action. Mackerel travel in schools and don’t like to think as individuals. Also, you need the rest of the people on the project to have such disregard for mackerel that they’ll agree to silently ignore management’s requests. If you have this perfect storm of “I don’t give a shit” philosophy you may be able to upwardly manage mackerel.

    Of course, if you find yourself in this situation, you might reconsider even trying this and either start up your own thing or get a better job where such absurdity is not necessary. Still, it can be a fun sociological experiment–especially if you don’t care about getting fired.

    VCs, Welcome to the Game Industry

    After my post about how Zynga seemed to have hit peak soccer mom, news broke that their profit took a nosedive by %95 this quarter. Some may say this is the bubble bursting on social, but it’s probably not. It’s just the maturing of the social game industry.

    Zynga’s active users are down by about 3 million. Growth is flat, this is true. However, Zynga has attributed their falling revenues to the lack of major releases generating new revenue. I’m likely to believe it.

    This is a familiar pattern to anyone who follows the dinopubs–companies such as Take 2 have wildly gyrating stock prices due to the release patterns of tent pole games like Grand Theft Auto. With AAA console titles taking 3-5 years to develop, you can go many costly quarters without any decent revenue coming in. This is especially scary when you are spending $50 million to develop a title and $100 million to market it.

    Remember the early days of social games? Think back to 2008 when Jetman was hot. It was a crude Flash action title that was probably whipped together in a weekend. SGN bought it as one of the first acquisitions in the space. Contrast that with Adventure World–a lush, rich adventure in…uh…clicking on lots of stuff to generate spam. Ok, not my cup of tea–but the amount of work put into this generation of social games is deceptively large.

    Zynga has been spending a lot of money acquiring companies and massive amounts of staff. I’m sure their 2,000 or so employees aren’t sitting around filling out Netflix trial subscriptions for free FarmVille Bucks. There’s a lot of development going on. Social games are taking longer to make and fickle audiences are saturated with compelling content. Naturally, if you don’t have anything new out there your usage and revenue will drop off.

    The thing is, this hit-driven pattern was part of what made games totally off-limits to investors until the social revolution. No longer can you throw a heap of money at a bunch of pizza-eating Stanford CS grads in a SOMA loft to crank out code and buy millions of users until you supernova into an acquisition by the greater fool. VCs wanted to invest in so-called ‘game factories’ but now you actually have to produce quality games and work hard for your users. Scary stuff.

    I’m not quite sure the crop of VCs that heavily invested in social gaming are ready for slow growth and a revenue chart that looks more like a string of al dente spaghetti than a hockey stick. Will social gaming become as unattractive to VCs as the traditional dinopub industry is? Zynga better not delay their IPO much longer–or time it to coincide with the release of a major title.

    The War for the Core Gamer

    Analysis of Zynga’s IPO seems to reveal that they’ve completely saturated the casual gamer market. User growth is flat but revenue is up. Luckily, they’ve been able to squeeze more money out of the their audience of the easily amused to make up for stalling player growth.

    These stats aren’t unique to Zynga. A recent presentation from Kontagent validates this. Across the board user retention is down along with virality. However, game session lengths, user lifetime, and ARPUs are all up. You’re getting fewer users staying to play but the ones left are spending more time and money in the games they like. Unfortunately, the cost to acquire these users is rising as everyone is fighting over the same market of bored middle-aged soccer moms.

    That’s why many social game developers are moving the war to a new front: the core gamer.

    Companies making so-called ‘hardcore’ Facebook games such as Kabam have shown you can extract more money out of a smaller audience of fanatical players. However, the f2p PC RTS League of Legends from Riot Games is a better example of social game economies successfully applied to the core gamer space. After all, League of Legends is a game that requires you to actually play it. Whereas many of Kabam’s offerings appear to be deeper, more complex spam hamster wheels that we’ve already seen before on Facebook. (Interestingly, Kabam has recently entered the social RTS space)

    This is where dinopubs such as Activision may have an edge if they can effectively apply their expertise in marketing and developing for the core market to the mobile and social space. Retail game sales have taken a dive, consoles have clearly been disrupted, and a new generation of gamers are emerging in the era of the tablet. Meanwhile, scores of Modern Warfare players are under served by throngs of Silicon Valley game developers cranking out isometric bakery simulators on a weekly basis.

    The opportunity is real. Which is why big players from outside the traditional game industry are gearing up for an attack directly on the core gamer segment. There’s a chance that the collective efforts of social game developers addressing the core space may make a serious dent in Activision or THQ’s growth by the time the next generation of consoles stumble out of the gate. Instead of sticking their head in the sand, dinopubs need to scramble to meet this challenge and create new revenue streams from the next generation of core gamers.

    Your Invention Assignment Clause Is Killing Your Company

    With unemployment at near apocalyptic levels tech companies still have troubles hiring programmers. In Silicon Valley the war for talent is intense. Word on the street is that Zynga is acquiring companies not for strategic value, but because they don’t want competitors to have the engineers.

    Salaries in the deep six figures, fat signing bonuses, and generous referral fees for those who have leads on a high a value target are put out there as incentives for skilled people to join the crew. Stakes are high.

    Part of the reason for this is that the talented, passionate, and driven simply don’t need a job. There are lots of ways to make lifestyle money in the tens of thousands a month publishing apps. Or you can raise $100k in seed capital for your nascent startup during a short cab ride. Why get a job?

    One of the new gimmicks is companies claiming they have an entrepreneurial environment in an attempt to bring reluctant candidates in from the cold. There’s one way to know this isn’t true, the “Invention Assignment” clause in most employment contracts.

    I’ve seen all kinds. Some are quite lenient and agreeable. Most want to own the contents of your brain from the day you start working until you leave. Even then, they may claim ownership of the few lingering pulses coursing through your neurons fired off during the final minutes of your employment. (Luckily, in some states these aren’t enforceable).

    If companies want to hire creative entrepreneurs they have to offer upside–risk, even. Let people take ownership of their ideas and responsibility for their success (or failure). The fact is the smartest and most creative employees won’t offer up a single decent idea if they know it’s company property whether or not it’s used.

    This requires forming an organization to incubate ideas instead of just bringing on new talent to your existing corporate morass. Create a place where talented individuals can thrive, instead of slowly morphing from a motivated entrepreneur to a veal in a pen fattened up on 3 hour meetings and regular paychecks.

    Unity3D Pet Peeves

    I’ve been using Unity3D a lot lately and naturally have developed a list of gripes. Let me clear my throat…

    The GUI.

    Having fluid 3D graphics in a Unity3D game isn’t a major accomplishment. It’s when a game has a great GUI that Unity developers crowd around and wonder how it was done. The fact is, Unity3D’s GUI system is absolutely atrocious. Getting anything decent out of it is worthy of a Congressional Medal of Honor. I shudder to think how much work it took to craft Battleheart’s amazing GUI in Unity3D.

    Back in 2005 when Unity3D was preparing for its first release, Immediate Mode GUIs were a hot new trend. However, game interfaces can be complex beasts. Especially on iOS platforms, there’s a certain level of polish expected that Unity’s IMGUI implementation can’t come close to. Also, Unity3D’s GUI has weird performance issues on mobile devices that makes it pretty much useless on iPhone.

    Unity3D 3.5 supposedly has an all-new GUI system that I’m eager to see. Unity hasn’t uttered a word about it. Instead they have been going on about AAA console game features such as light probes that are largely irrelevant to anyone doing mobile games.

    AnimationEvents can only be called on the first script.

    Adding and manipulating animations in Unity3D can’t be much easier. Just drag and drop animations on to a prefab. Blending between animations is just a simple API call. You can even point and click to add AnimationEvents on any animation’s timeline. These are function calls that are made on the timeline, usually to trigger sounds, particle effects, etc.

    The problem is, the AnimationEvent system can only call functions on the first script component attached to the object. This wouldn’t be a big deal if you could re-order the scripts in the Inspector. Instead, the only way to change the script order is to remove all script components and re-add them with the one containing AnimationEvent code first.

    Asset Server should merge binary objects.

    Before Unity3D supported source control in the Pro version, your only option was the $500 a seat Asset Server. This is a no frills source control system that allows multiple users to collaborate on scenes together. Unity should be able to craft a source control system that knows its own object format. If you could merge changes of binary objects down to the component level, Asset Server would be invaluable. Otherwise, Perforce is a better alternative. Even with Perforce, scaling a Unity3D development team beyond 10 or so people can be a nightmare.

    Why are you using Unity3D at all?

    Here’s the thing. Why are you even using Unity3D? Is it for 3D graphics? Who cares? There might be one or two 3D games in the top 50 iPhone apps sorted by revenue. On Facebook, nobody cares about 3D either.

    The fact is, Unity3D is busy trying to attack the collapsing AAA game market with high end features such as multi-core renderers and light probes while ignoring what the vast majority of their users need–mobile friendly features such as GUI and 2D support.

    My interest in Unity3D stems from its multi-platform features. The ability to effortlessly spit out web, iOS and Android builds from the same code is attractive. Still, when %80 of your revenue comes from iOS, do you really think it’s worth the hassle of dealing with Unity’s shortcomings just to scrape up a few extra dollars?

    The amount of time I’ve spent searching for the ultimate cross-platform solution would probably have been better spent nailing an excellent iOS version and dealing with other platforms later. Still, in the past I’ve been bitten by tying a codebase closely to a single platform when the market shifts. This is an open question, but 3.5’s new GUI may be the make or break moment for my Unity support.

    How to Talk to Rich People

    At certain times in your adventure as an entrepreneur, you may have to talk to the rich and delusional. As many you know, I advise teaming up with someone who specializes in this. Just in case you find yourself unarmed in the presence of a rich guy who is looking to invest in your start-up, here are some tips learned from observing others much better at this than I am.

    Oh and, when I speak of weird rich people, I’m not talking about VCs or experienced angels. Most of these guys at least have a semblance of a process and logic behind how they operate. I’m talking about loaded dudes who know almost nothing about your business but are looking to get in on the tech bubble.

    Anyway, those tips…

      Ask for something. Have a reason for the discussion. A lot of super rich dudes are just lonely. They break up their solitary existence by inviting you on their boat to have long, rambling, pointless discussions. Have an agenda and don’t be afraid to ask for what you want. Otherwise, you’ll spend hours going in circles.
      Never say no. When you reach a certain level of wealth, you stop hearing the word no. When talking with such people, disagreement is considered “resistance” and jeopardizes the progress of your conversation. You might just phrase “no” in a softer, more creative way. Unless he asks you to drown a basket of puppies or something. In that case, look for a life raft.
      There is no problem that can’t be solved with the simple application of more money. The idle rich do not want to hear about technical limitations of your product that might make their input unfeasible. Worst case, it’s an issue–but can simply be solved with more money. We know this isn’t true–after all, we build stuff. However, being realistic will paint you as a naysayer and killer of dreams. Keep the reality distortion field powered up.

    For many, these talents are natural. For nerdy programmer types, talking to money men is usually a learned skill. It’s also a full-time job–you can’t develop a product yourself and raise money simultaneously. Find help.