Merry Xmas - Top 5 testing insights in a box!

Santa pulling faces

Santa pulling faces

It’s that time of year again. Santa’s are appearing everywhere, Christmas trees are going up, houses are lighting up like mini constellations and Christmas Jingles are - for a few days at least - relatively okay to listen to in shops.

And of course it’s gift time too - socks, ugly jumpers and box-sets of Transformers for everyone!

Unless, of course, you’d prefer something a little more useful.

For those of you not familiar with us we use a Research Management Platform that allows us to track issues over time. It’s called FlexUX and it’s free to our customers. On their behalf we’ve researched everything from Intranets to corporate websites, e-commerce stores, sign-up processes, campaign sites, apps, Enterprise apps and physical devices (BBQ’s and IoT devices were the most fun this year!)

So we’ve created a simple comparison across all of the user tests we’ve run throughout this year. We’ve pulled in all of the findings across priority and category encountered, and compared them to 2017 data, just for fun. Of course this is a minor data set (under 40 tests per year, average 11 people per test) and its self-selecting in that it represents the people who decided to test with us - but it’s still interesting.

And here are the top 5 comparisons we can show.

1. Someone’s been naughty this year….

The very first day I decided to start my own UX consultancy I remember having a conversation with a designer who told me I was crazy. When I asked why, he said “Because we’re learning more and more about what works - in a few years time we’ll have simple guides that will have everything written out and there won’t be any need for UX people.”

In theory we’re all getting smarter every year. and whilst my career choice would have been shot I would have loved to have been proved wrong.

But even so we don’t seem to be getting any closer.

If we were learning everything about design then UX issues should be both decreasing in number and decreasing in severity. Or at least one of the two.

Unfortunately that’s not the case, based on our FlexUX reporting. 2018 shows an almost depressingly identical mix of issue levels and severities, as a percentage - see graph below.

Graph, findings by priority, year on year

For anyone who’s not aware we categorise UX issues by High, Medium and Low (Positives are the things people love about a design, not issues but lovely to record).

  • HIGH - Potential to stop the user achieving their goal

  • MEDIUM - Potential to confuse and/or slow down the user in achieving their goal

  • LOW - Doesn’t slow down or confuse the user but impacts their positive experience

There has been a very minor shift from High priority to Medium priority this year overall - but not much. Someone’s not on the Nice list…

Graph, findings by category, year on year

2. Down down, Content is Down

We categorise all issues we encounter into a standard UX system, within which ‘Content’ issues usually make a strong showing, especially in the Medium and Low priority issues. Content issues are issues that involve:

  • Poorly written or delivered content

  • Missing content (users want a piece of information that isn’t there)

  • Too much content (getting in the way of the experience)

  • Typos and mistakes (usually within prototypes)

This year we have, thankfully, seen a good drop in Content issues, dropping nearly 10%. In general we see a higher level of focus on content, which is great - getting customers to the right point is fantastic, but if the delivered experience at that point is poor then all is for naught.

So well done all the Copywriters and Editors and Reviewers we’ve worked with - your good work is paying off, and Santa knows it.

3. Sometimes not seeing something can be a blessing

Said AUGUST STRINDBERG, apparently. But in general, we want our users to see our work.

Unfortunately the percentage of people who didn’t see went up this year, by a few percent.

Visibility issues generally relate to:

  • Not seeing something that the user is looking for (but it’s there, somewhere)

  • Focussing on something they don’t want to see, that gets in the way (hello, Ads)

  • Not being able to clearly see something they need to see (poor contrast in fonts or images, for example)

Whilst designs have gotten sexier this year, unfortunately that hasn’t translated into designs where it’s easier to see what we need to see. That’s naughty, not nice.

4. Not finding things, sucks. Still.

Navigation/Interaction issues encompass a few different areas, including:

  • Not finding content (whether navigating or searching)

  • Finding it but only after going to the wrong places

  • Struggling with the interaction model (now much rarer, in general)

Information Architecture is one of the most basic skills in the UX space and it should be an embedded component in all projects.

Unfortunately the results are not showing that to be the case. I suggest that’s partly because people think Search will pick up the slack and people will find everything that way (Everyone uses Google anyway now, does anyone even use the nav bar? Answer, Yes. Yes, they do.) But it’s also because not enough thought is being put into ensuring the structure of content aligns with the expectations of the audience.

No stocking over the fireplace for you people.

5. Like a good marriage, good design is all about how you communicate

Feedback issues are about the way a design communicates to the audience. These issues generally fall down to:

  • Failing to tell the user what is expected (e.g. hints on forms)

  • Failing to explain to the user what happened

  • Communicating poorly (wrong tone of voice, confusing wording, etc.)

Feedback issues have almost doubled in 2018, and this is sad to see. Whilst by far the biggest percentage of issues relate to content and navigation, feedback issues are fast coming up on third place.

A lot of these issues come down to pure laziness - not planning out and reviewing error messages, not checking on hint text, not bothering to explain what a function does (because users will just know). But others come down to minimalist design and a love of clean pages - at the expense of explanation. Placing hints and/or labels inside fields (so they disappear as soon as the user types), or having clever interactions to show help that the user never learns.

That’s a gift that nobody wants.

So, 2019?

We’ll be running FlexUX throughout 2019 so hopefully we’ll be able to report on progress and see how UX results are shifting.

We should be testing more fields - VR and AR, Voice Systems and AI-based interfaces. It’ll be interesting to see if they’ve learned from Santa’s Naughty List - or are doomed to repeat it.

Let’s hope for the former.