Susan Farrell on Decreasing Mozilla Support Costs

A redesign of Mozilla’s online help system used input from user behavior (e.g. search terms) and forum questions. The effort paid off by greatly reducing the number of support requests in a short time, and by allowing staff to respond to almost all support requests within 24 hours. A post by Susan Farrell, of usability consulting firm Nielsen Norman Group (NN/g), includes nice graphs of both payoffs.

If you’re not familiar with NN/g, that’s “Nielsen” as in the author of Designing Web Usability and Mobile Usability, and “Norman” as in The Design of Everyday Things and Turn Signals are the Facial Expressions of Automobiles. In the spirit of under-promising, the partner roster also includes Bruce Tognazzini (ex-Apple, author of Tog on Interface and Tog on Software Design).

At the end of summer 2011, Mozilla staff received over 11,000 user questions per month. Why so many? Clearly because many users are asking these questions! But beyond that, why do users need to ask the questions they do? Farrell identifies several issues:

  • 400 pages of online documentation were difficult to search.
  • Time spent fielding user questions took staff time from writing new help files, or improving old ones…
  • …But on the other hand, the accumulation of the new articles staff could write “caused more findability problems.”

» Read more

Aaron Rudger’s “Three Ways to Captivate Customers on Wearables”

This is how it happened, more or less: My colleague Gilberto Castañeda asked what I thought of the SQE/Techwell STAREAST Virtual Conference.

“Oops! I missed it! What was good?”

While I parsed his response to this question, I groped for an algorithm to choose which archived session or sessions to view myself. Since Gilberto hadn’t had much time for the conference, this suddenly got easy.

“Of the sessions you didn’t see, which interested you most?”

I thought I could feed the network by viewing that presentation, and providing a summary; hence, this post. Turns out Gilberto was most curious about Aaron Rudger’s presentation, “Three Ways to Captivate Customers on Wearables.” The synopsis:

Competing for growth is a game of seconds. Superior customer experience is the #1 differentiator for digital success today. Wearable devices and the apps that power them promise to change consumer life and employee productivity. Developers and Quality teams must meet the demanding needs of users whose expectations for a flawless experience are changing.

Aaron Rudger will discuss the three keys to success for delighting customers on smartwatches, smartphones, tablets or digital form-factors yet to be imagined. Aaron will share testing best practices to prepare your apps for a smooth and reliable experience now and in the future.

Buzzword alert! But what else would you expect from a presentation on wearables? Anticipate people who are racing to establish themselves at the forefront of an emerging technology composed of both agile and massive corporations racing to establish themselves…. A conference speaker on this topic will be surfing on someone else’s bow wave.

Rudger set the scene with rhetorical questions: How do you get high quality results from testing wearables? How do you ensure a high quality user experience?

Rudger equated three models of Apple watches with “Three Horsemen of the Testing Apocalypse.” Smart watches are not a new idea, but change the testing game. These will create a new paradigm for engaging with customers, Rudger said. (Really? There it is: the market for testing software on smartwatches boils down to those who want to use the watches to sell stuff. A friend shared his dystopian reaction while I wrote this.) There will be great pressure to deliver these new experiences.

But Rudger really isn’t here to talk about the scope of wearables. He’s not interested in sensor networks woven in fabric, for example. In fact, he’s drilling right down to the Apple watch. Why?

“It’s all about the money.”

It's All About the Money

Oh, right: that! Well, then! Let’s get to it.

720,000 Android watches (or “wear devices”) were sold in all of 2014. In contrast, an estimated 1.2 million Apple watches sold during the first weekend of their availability, with 2015 sales forecast at 20 million. Rudger forecasts linear growth, quickly leading to a multi-billion dollar market with concomitant pressures. He also cited a prediction that 2.5 billion smartphones will be active by end of 2016.

Building and running apps for the Apple Watch dependent on Apple’s WatchKit. Even if you don’t develop functionality specifically for the Watch, developers will take advantage of its functionality, and of WatchKit. WatchKit affects mobile developers “whether or not they’re in the…game.”

Rudger asserted that the Apple Watch has driving the doubling of iOS release velocity.

By the way, don’t expect Google to sit still and watch while this happens. Android will continue quick evolution.

Power management is critical to apps for wearables; functionality and performance blur in this domain. Quality is tightly coupled with performance.

Rudger said that conventions in this domain are not well-defined. Really? Is this an opportunity for someone with a big mouth? Can’t one work from first principles and by analogy to establish a good first draft? What does usability god Jakob Nielsen have to say? Or Don Norman? Bruce Tognazzini? Has the smartwatch caught people like these flat-footed?

With a perfect sense of timing, while I was writing this post, Nielsen Norman Group Senior Researcher Raluca Budiu weighed in with “The Apple Watch: User-Experience Appraisal.” Her article cites an earlier look at the Samsung Galaxy Gear. Budiu describes Apple Watch GUI successes and failures with reference to broader principles, easily translated to design guidelines. I think we can say that, no, the usability crowd has not been caught flat-footed. But they can’t tell us what the smartwatch killer app is, either. Budiu’s conclusion is to offer some guidelines for those with the nerve to believe they can actually create value for smartwatch users. She can’t tell them where this value lies; only what its delivery vehicle should look and feel like.

Not long after, I came across Nielsen Norman Group principal and usability god Bruce “Tog” Tognazzini’s current draft “First Principles of Interaction Design.” This document also addresses smartwatches.

There may be a cultural or psychological factor at work in not acknowledging these or similar resources, e.g.: “We’re a startup. We don’t have time for that.”

And that brings us back to Rudger’s presentation: How hard will it be to develop good smartwatch apps? What must designers and developers consider? “…The application and experience is going to be contextually based; it’s going to require an extreme amount of curation….”

Does that help you? Me, neither.

In fact, the thrust of this section of the presentation is to reiterate what people have probably been telling each other since at least the invention of written language: Everything is faster and more complex than it’s ever been before. Only the fleetest of foot, Mercury-winged, have a prayer of even standing still (in Jeremy Scott’s styling footwear, no doubt).

Or is it completely true, and I’m just bored with the sound of being swept into the crowded dustbin of history?

But wait: Here’s the plug, and the reason Keynote LLC was happy to have Rudger, its Director of Product Marketing, spend time on this presentation: All of this means we should test in the cloud.

Cloud? What cloud? What does this mean? That we should use email and FTP to work together? Or that we have emulators available online? Oh, good: Here comes a well-timed demo.

Yes, they’re emulators. Not just emulators: emulators developed by Keystone! Like the ones I collected a dozen or two of when developing wireless applications 15 years ago, except that these run on someone else’s hardware, somewhere out there. They’re controllable through a web client.

At least that’s what the naive viewer might believe. But wait! He would be wrong! These are not emulators; Rudger’s showing us an interface to real hardware! (A free trial is available at mobiletesting.keynote.com.) Having multiple devices available at the same time enables easy exploratory testing. One device can send a message to another.

Rudger suggests that a great virtue of this approach is that it does not burden developers with the need to learn new languages. However, it’s amenable to both manual and automated testing (presumably through approaches like Selenium). The approach immediately supports new operating systems.

After about 25 minutes of introduction and presentation, we segue to audience questions and Rudger’s responses. Participants can’t help but feel rewarded for their efforts: Rudger thinks every question posed is “really great.” One that stood out:

Q [paraphrased]: Are wearables a fad?

A: No, because Apple’s involved.

Is that the deepest Rudger can go, when he could assemble a position using components like Moore’s Law?

Q: How many users are on a particular hardware device?

A: Capacity is adequate, and continuously monitored. Keynote adds devices to accommodate contention. There’s a careful process at work here. Of course there is. With any luck, Keynote won’t be as investment-averse as some of its clients are. Don’t make that process too careful!

Your Cloud Testing Environment Can Host Me and All My Friends, Right?

If you want to see for yourself, you can find Rudger’s presentation online until 8/7/2015. (You’ll have to register.)