In our most recent UX Insider interview I had a great chat with David Hamill, a usability research expert at Skyscanner.
Firstly, could you give me a bit of a background on Skyscanner and what you do?
Skyscanner is known for being a flight comparison website. We also compare hotels and car hire, and provide that on our website and via an app. We have really good traffic. More than 40m worldwide.
We initially started as (what’s classed in the industry) as a ‘meta-search’. When searching for flights, if you go to Expedia for your search, the site is just a big travel agent, so their main objective is to actually sell you flights and extras, whereas we’re a comparison site and people don’t tend to realise this which is a UX challenge for us.
And what is it that a usability specialist at Skyscanner does. Can you explain your role?
My title these days is ‘User Researcher’. My role is varied; I undertake user research, but I also help the designers, or rather the squads, to do their user research. Our organisation is broken into squads, and also our products are broken down into these squads. We follow the Spotify model of using Squads and Tribes. It’s all about speed of delivery, and quality of delivery. A lot of Internet companies will be using squad models these days.
I’ll do research for them, but in some cases I won’t because it’s difficult to research a single page. Instead, I’ll help them with some smaller, tactical things, and facilitate these squads to be able to do quick research themselves. They’ll all run experiments, and test their hypotheses as well.
I try and get squads to take ownership of findings, and facilitate the way in which they can get to that point. This is quite a new direction I have taken. I have to reprogram other people’s thinking; they’re used to someone else running off the user tests, and returning with the information on the bigger user pain points, with recommendations on how to make changes.
I also generally ‘agitate’ the company on subjects that come up a lot from research that I feel aren’t quite getting the attention that they deserve. But – it depends on what the issue is, and what is the best way to sort it. I say ‘agitate’, but it’s probably more ‘irritate’ as I am shaking things up!
When we’re talking about you doing research, are we talking mainly user testing? Are we talking remote testing, or face-to-face?
I’ll use the quickest and easiest method to get the best answer, and then progress from there. If I’m using tools for tactical purposes, I’ll use UserTesting.com, or WhatUsersDo, and see if someone from a panel on these websites understands how it works. If after that I feel that we didn’t get the desired answer, or more work is needed, I’ll carry out some user testing.
People tend to have a ‘dream UX process’ in mind, that never ever happens in its purest form, and because of that, the part that we can improve on is to do more research up front.
At Skyscanner, we’re very engineering-led, so a lot of people jump to building first. Then there is not as much time for UX research.
Here’s a talk David did at NUX2 titled ‘A random collection of tips about running usability tests’
Can you tell me about the challenges that you’re working on at the moment?
Right now, I’m wrestling with – for want of a better term – the ‘experience model’ of how people plan trips. It doesn’t exist! Or if it does, the model is very difficult. In the last two weeks I’ve completed around 15 user tests where I, or another moderator, let the users (who are genuinely looking for a trip) search for a trip, as if they would on their lunch break.
We want to see the different ways people address the problems when planning a trip. I’m also looking at how people decide upon the destination of their trip.
This is quite challenging. It’s more of a background project, as it’s been ongoing since September last year. Whenever I find the time, I’ll dip back into it.
I’ve been chatting to about 30 internal employees who hadn’t decided where they had wanted to go – they had a trip in mind, but no destination. I just keep in touch with them and see how their journey unfolds.
“The mistake that I made a good few years ago, and see other UXers making all the time, is to think that there’s always a business payoff for your UX”
What would you say is the biggest UX mistake that you’ve ever made, and what did you learn from it?
The mistake that I made a good few years ago, and see other UXers making all the time, is to think that there’s always a business payoff for your UX. It’s a very easy and comfortable place to sit, when you say: “You need to fix that. If you fix that, your business will improve.”
But the reality can often be the opposite. So if you’re in a place where you don’t need to prove it, it’s very comfortable to be able to say that, because you’ve never been challenged on it.
But here, I have made recommendations that have caused critical incidents in this company – you have to learn from that and prove that something has an impact before you implement it. It’s not enough to just rely on qualitative user research.
I identified something that seemed to delay nearly every test user. It caused a slight bit of friction, and every time I got a user in, I’d be seeing this friction. They’re trying to do something, and we’re sort of diverting them away from it.
Before we had our experimentation capability, the people in charge of that product said: “Okay. Well. David says that’s going to be an improvement.” I gave some video evidence of it happening rather a lot. So we changed it.
And then, our conversions started going down! We couldn’t work out why, so we ended up addressing a whole load of things, but it was actually that thing that I recommended that we do….I still really don’t know the answer to this day. I have hypotheses about why it is, but I haven’t proved it.
So how do you avoid that happening again?
Well – we don’t put out anything that doesn’t go through any sort of experiment, and we’ll experiment with everything that we release.
When you say ‘experiment’, do you mean A/B testing?
Yes. A/B, or A/B/C/D Initially, we release it to a very small amount of people for 24 hours, to see if it would really tank things. If not, then it gets opened up.
Depending on the impact that you see, maybe 10% of the UK, or 10% of worldwide will see it, and we’ll watch it for a week or two, to make sure that what we say is going to happen is definitely going to happen.
We don’t want the sort of technical overhead of supporting things that don’t really have an impact.
“The first thing that I did was to go back through the hundreds and hundreds of complaints, and categorized them all”
Do you have any examples of any recent changes that you’ve made on the back of user research that has had a positive impact?
It’s very hard to choose! I’m very proud of the company, I shouldn’t claim too much credit for it – I’m very proud of the improvements we’ve made on our map. This is something that we hid away as an advanced feature. We did have a map that was quite hard to use, but it allowed people to do stuff that they struggled to find elsewhere. We tried removing it, and received a barrage of complaints from people who were dependent upon it, so we put it back and tried to improve it.
The squad that looked after that was very user-focused, and did a lot of improvement research. The first thing that I did was to go back through the hundreds and hundreds of complaints, and categorized them all to see what people were trying to do with it. Then it’s possible to see that there are four or five key things that people are trying to do with that map.
Everything that the squad develops is based on extensive user research. The focus was on what users were trying to do with the map, and I think as a result it’s a good example of something that’s visibly changed quite recently in our product.
What would you say is the difference between somebody who is a good UXer, and somebody who is a great UXer?
I think somebody that’s a good UXer is perhaps a bit tactical, and good at spotting things about interfaces and products that could just be made a little better. I think a great UXer looks a wee bit further up the road.
I think it’s better for a user researcher to really concentrate on how a user behaves on a system tactically, trying to understand why the little interactions that they do is the way that they do it, etc.
I think a great UXer moves beyond the norm – I’ve just realised that in this conversation, I’ve appointed myself as a great UXer! What a big head! But a great UXer moves beyond that, without forgetting about it – and thinks about what that means further on.
You worked for quite a long time as a freelancer, and then you transitioned into a full-time role for one company. What was that like? What was the difference?
It was a lot easier than I thought it was going to be. I was lucky with Skyscanner, because when I was offered a job here, I was already the go-to person for this type of thing. I spent a lot of effort in trying to maintain a little bit of distance from people’s projects, so that I could get the same sort of objectivity to it.
I realised that I had to take on a role internally, unless I wanted to move out of Edinburgh, and I didn’t want to do that. And so, faced with the options I had, it was finance or this. Skyscanner made it very easy for me. They allowed me to come and work three months for them, still being self-employed, and then decide if I still wanted a job with them. So there was no need to then make a rush decision.
And if I didn’t want to work there permanently, they said: “That’s going to be cool, we’re still going to hire you. We’re just trying to find the best option of working with you.” So it was made very, very easy by Skyscanner.
“I understand way more about user experience on this website and others, because I’ve learned to embrace the numbers.”
What would you say is the most valuable thing you’ve learned while working at Skyscanner?
I used to be too qualitative data-focused, and so coming to Skyscanner allowed me to sort of fill in that part of my profile that was a bit missing. They are very quantitative data-driven here which has really helped.
I made a lot of recommendations to Skyscanner that were based on qualitative data, and I never had access to the analytics. Now I’ve got access to all of that, I understand way more about user experience on this website and others, because I’ve learned to embrace the numbers.
What would you say the hardest part of your role is?
Because I’m not attached to these squads, they are self-determined, as it were, so they’re given lots of freedom to go and work on their own things.
When I find stuff that I think affects them, they’re probably not in the right place to hear it, because they’re looking at a different angle, they’re looking at a different aspect. That’s difficult.
Also, just because I’ve found out about something, doesn’t mean that everybody will drop everything and listen to me. It means that I can be a bit of a drifter in here. You often don’t get the pat on the back for things. You know, you can point out how something could be made a lot better; they’ll kind of listen and work on it. Then, once the improvement goes through organisationally, they’ll get the merit for it.
They’ll be very appreciative to me, but you have to let go of those things, because it’s very easy to have the ideas. It’s very easy to find things in user research, and make recommendations. You don’t have to really stand by them if you’re not the person that then has to go and build it.
So if you imagine that somebody is going to start a role like yours in a different company, what sort of advice might you give them?
If they were looking to choose somewhere to work, I would suggest they look for signs that they are going to be able to make an impact, because if you’re not going to be able to make an impact, you’re taking a job in which you’ll eventually just go into autopilot. You’ll go through the motions of being a UXer, or user researcher, but ultimately some cultures can just drive the impact out of you.
I’d also say look at the organisation, look for examples of it. If they’re looking for you to be the person to explode apart this behemoth organisation that’s culturally set in certain ways, and it’s tied down organisationally, it’s probably not going to be a great job.
Thinking of the future, say in five years’ time, what do you think might be a bit different in the way that we, , plan and book travel?
Everybody’s wrestling with this in travel. I think there’s a balance. A lot of people think that we’re trying to take the chore away from travel booking, or travel planning. Some people think that Skyscanner should tell them which flight they’re taking. But there are people that find travel research very enjoyable.
If I just deliver to you the best flight, the best hotel, ‘now go and enjoy your holiday’, I’ve just cut off a large part of your holiday experience, because I’ve not allowed you to plan it. I think there’s a balance between striking out the stuff that’s laborious and horrid, but still giving you the experience that when you’re planning your holiday, you’re already on your holiday. That’s the way it is for a lot of people.
I don’t think, as an industry, that we accept that, or have identified that yet on a bigger scale.
Where do you go for inspiration?
I often see how Google are doing stuff, and then try to humanise it, because Google often have very interesting ways of approaching things.
I find that they’re often too logical for human beings. Google’s a good place for interesting things. I have never looked at something that Google has done and not thought: “I could improve that.”
We are trying to do more and more of these interviews. If you could pick anybody that you would want us to interview, who would that be?
For selfish reasons, I would pick someone from Booking.com.
I look at Booking.com, and I see a product that has been built through experimentation, as marginal increases have been made. So you get, as a site, a bit of a dog’s dinner.
I wonder what sort of impact and role user research in Booking.com has, because they seem to me to be all about numbers. And while a user researcher should be very comfortable with those numbers, they also should be very close to qualitative stuff as well, and I wonder how much qualitative research is done at Booking.com.
————
Thanks for reading. Following on from David’s interview we got in touch with a UX guy from Booking.com so look out for that one soon.
If you would like to be interviewed, or you’d like us to interview anyone in particular please get in touch
You can find David on Twitter: @dav_hamill , or check out more in our UX Insider series
