‚Äč

User research of Windows 8 makes for fascinating video

March 20, 2012 By Mark Hurst 14 Comments

Here’s something you don’t often see: more than a half-million views for a user-research video. But it’s well deserved. In How Real People Will Use Windows 8, tech blogger Chris Pirillo shows his father (60-or-70-something) using the release version of Microsoft Windows 8. It makes for gripping video. Like any good listening lab, we’re seeing a real live user experience: a person honestly try to derive some value from the product. We feel the dad’s frustration as he can’t figure out how to navigate away from a confusing display of app icons.

The irony is that the father is a longtime Windows user, and at the end of the video, he asks whether Microsoft is trying to get him to switch to Mac. (!) Given the trajectory of the iPad’s growth and people’s delight with it, versus Microsoft’s stagnation in the consumer market due to people’s continued frustration with Windows, his joking comment is partially true: Microsoft, despite itself, is launching a new product that will likely drive more consumers to buy a Mac.

Speaking of Microsoft, I have to wonder how (or whether) this video is making any changes there in the mindset of the Windows product managers. Did the company conduct non-directed listening labs on the product, see this utter failure of the UX, and still approve the launch of such an obviously problematic interface? Or did lower-level product managers know about this problem all along but were outvoted by higher-up execs who just needed to ship something, anything, to meet a launch deadline?

COMMENTS

  1. Howard Greenstein March 20, 2012 8:20 pm

    Mark,
    I’ll admit I had the same experience, and I consider myself pretty savvy. I also have a windows phone to play with, so Metro isn’t tough for me.
    But finding start (it is where it used to be, just “hidden”) was confounding until I searched the net for it.
    As a former Microsoftie and shareholder, this video makes me furious at the UX team.

    • Mark Hurst March 22, 2012 12:49 pm

      Thanks, Howard. I wonder how many users will want to search the net to find basic functionality in Windows 8 :)

    • Stephanie April 03, 2012 1:08 pm

      In my experience as a UX designer it’s really the Product Managers who drive the actual design of a product, we mainly implement what the rest of the team instructs us to do. Sure we present research, argue for usability, and defend our design work, but other team members typically hold more sway over what gets released. Mark, I think your comment at the end of your post is very telling and I would like to know too. “…did lower-level product managers know about this problem all along but were outvoted by higher-up execs who just needed to ship something, anything, to meet a launch deadline?”

  2. BrynDews March 22, 2012 3:09 pm

    Mark,
    I am betting on the latter assumption in your closing. I say this based on some “insider information” … a conversation I had with the head of UX at Microsoft way back when we were participating in a Beta of Office 2007. This guy said that the Marketing wonks pretty much said they didn’t care that the pretty little windows icon was confusing the hell out of the users… it MUST be there for branding! Note that Office 2010 reversed that incredibly poor decision and put the “File” menu back! But, how many users did they alienate (are still alienating since so many people couldn’t afford to keep ugrading office with every new release) while it was out there?

  3. George Girton March 22, 2012 3:16 pm

    Only because I remembered that “control-escape” brought up the start menu was I able to start using Windows 8 Consumer Preview at all. (I was one of the half-million viewers and, to be fair, I didn’t watch the “how to” videos on the Microsoft website before downloading, installing, and actually using Windows 8 successfully). Other than my eventual success, my experience had similar aspects to Chris P’s father. My assumption is that Microsoft are NOT DONE with the desktop product — but then I read David Pogue’s review saying how wonderful it worked on a tablet. As they say in the biz … good luck with that.

  4. Mark March 22, 2012 3:50 pm

    So . . . how DO you get back to the boxes?

    • Mark Hurst March 22, 2012 6:42 pm

      I don’t know.. I think there’s some unusual gesture one has to do with the mouse.

  5. Bill Koenig March 22, 2012 4:05 pm

    I have been using the pre-beta since it came out, and am now using the beta.
    Yes, it is different, but you don’t have to use those crazy icons. I use the classic and get along very well with it. It is nothing but Windows 7, with a little bit of dressing. As with any new program, you gotta work with it to learn it.

    • Mark Hurst March 22, 2012 6:45 pm

      Thanks, Bill. Given that you acquired and installed the pre-beta, I’d guess that you’re a little more technically advanced than the average user. It’s good to hear that – for advanced users – there’s a way to get the UI back to a usable state.

      The more pertinent question, though, from a product viability standpoint, is – what happens when the vast majority of users encounter this UI? If they are stopped dead in their tracks like the user in the video, Microsoft could have a significant problem on their hands – potentially requiring an expensive fix.

      If that’s the eventual outcome, I hope Microsoft will conclude that it would have been far less expensive to do a little bit of objective, non-directed listening labs before launching such a difficult UI.

  6. Chris Barry March 22, 2012 6:57 pm

    I’m going to bet/hope that the OS will come with a little introduction when users get it. Keep in mind that its not exactly meant for everyday users right now, but rather those who are technically savvy enough to be able to install an OS without being concerned… probably not Prillo’s father. And Microsoft probably expects those users to at least watch its videos before downloading

  7. Deb Well March 23, 2012 9:34 am

    Expecting users to watch a video before downloading and using your product does not speak well of out-of-the-box usability. Average folk don’t have the patience for that. And of course, advanced users don’t want to mess with that either – they figure since they are advanced, they will be able to “just figure it out”. Which is not to say they won’t get frustrated along the way.
    I am Microsoft trained, and when I first sat at a Mac, I was frustrated because I didn’t know where everything lived underneath the interface, the way I did with Windows. But I was pretty quickly won over by the fact that Mac’s usability meant I spent more time actually doing what I wanted to do, and far less time on overhead, administration and technical issues. And am happier for it.
    It baffles me how, after all these years, Microsoft doesn’t get the usability concept.

    • Mark Hurst March 23, 2012 9:59 am

      The irony is that Microsoft invests a *lot* in usability – employees, physical lab space, etc. – so I wonder where the breakdown occurs. Are the UX methods they use outdated and ineffective? Or is the UX work excellent but simply ignored by the political structure within the company?

  8. ClasGuit March 27, 2012 11:31 pm

    True there is some learning involved in working with Windows 8 compared to its earlier versions. But once we understand that the 4 corners on the monitor is a way to get in and out of metro , it is so much more better. It is great to have Metro and Desktop power on the same machine. There is no alternative to learning. Just like when a person in Europe learns to drive on the wrong side of the road while driving in US. Just like we all learn to try and eat with Chop sticks when we visit Chinese restaurants.. this too is an opportunity for us to learn the Metro and Desktop world of Windows. Because my old man doesn’t know how to eat with Chopsticks , I will film him struggling to eat and post a video saying Eating with chopsticks fails dad’s test , is the wrong way of looking at things.

  9. Matt April 09, 2012 11:54 am

    Interesting… however I would like to see a video of the same guy using an iPad or an Android phone (or god forbid Ubuntu.)

    Then let’s watch another video of my kids using Windows 8 for the first time.

    This kid is “trying to prove a point”… it was a good try.

    (Disclaimer, I own Windows, Mac, Android and Linux products and I like them all.)