Building a great digital experience is all about understanding your users. We’ve been beating the drum about how agile airline UX research should be the core element of your optimization process even way before COVID-19 hit us.
I’ve written about how Ryanair is doing airline UX research at scale and how Virgin Atlantic built their user-centric digital optimization team.
However, one can argue that COVID-19 changed your users’ behavior so much that you must start your airline UX research activities from ground zero. Even if you think that’s too drastic, at least re-think what kind of user and UX research activities you do.
Amanda Stockwell, UX research expert, LinkedIn teacher, and Diggintravel Airline Digital Academy Instructor, certainly advises you to re-think what kind of activities you do:
In this time where needs and contexts are shifting so rapidly, you don’t need to know how easy or not it is to click a button or to get through a specific process. You need to understand how people are conceiving and how they’re feeling, and you need to be able to come up with ideas to address their biggest challenges and that’s preventing them from interacting with you.
I talked to Amanda on the last episode of the Diggintravel podcast. You can listen to our full talk via the audio player below, or read on to see key tips for your airline UX research activities.
The first thing Amanda advises you to do is to understand your goal and the context before you select what kind of airline UX research you’ll do:
I would say the very first thing to look at is, what is your context? What are you trying to learn? Try to narrow down some specific goals. For instance, if you are exploring building a brand new product, you’re going to want to do something really differently than if you are iterating on an app that’s been out for a really long time and you’re just making small changes to it. So the first thing is to think about the context of what you’re working on.
Are you building something new? Are you assessing something that’s live? Are you assessing a new idea or a new version of something? Or are you trying to learn more about your users and overall context? Take that breakdown first. Then look at the specific goals and specific things you’re trying to learn.
The next step Amanda recommends is to think carefully about what kind of airline UX research methods you’re using. To select the right method, especially in the current time, you need to think about what you’re trying to achieve. There are many different methods, and if you don’t have a good understanding you can easily get overwhelmed.
Here is a great overview of different methods by NN/g group:
Source: NN/g group
The trend that I see is that we think the UX research methods that have become more accessible with a lot of new tools, like web tools to do session recordings, analytics, or heat maps, are enough. Here is Amanda’s take on that:
It’s not that those things aren’t useful; it’s just that they won’t give you the full story. They won’t lead you to any new opportunities. You might be able to refine your current offering or your current product and make that as good as possible, but it won’t lead you to insights about where else you should go or how else you could serve.
So, analytics, user flows, session recordings, and unmoderated user testing work great when you’re evaluating how your existing user experience works. However, these quantitative methods are not the right way to go when trying to understand a change in user behavior or when you want to build new solutions.
Amanda recommends that because of COVID-19 and the shift in behavior, you need to use more qualitative and exploratory methods:
The bigger value from a business strategy perspective is to do user research that’s generative and qualitative and exploratory to better understand the shifting context.
And given that we are speaking in August of 2020, I think that it goes without saying that things change rapidly. Anybody who was asked 6 months ago what they would be doing right now probably is not on the exact same track they thought they were going to be. With the shifts in the world and technology and global pandemics, really doing ongoing user research to understand your users and their shifting contexts and needs can help you adapt and really find new opportunities to serve and provide value – which ultimately is going to build brand loyalty.
Based on all the things she’d said, I asked Amanda what kind of airline UX research method she would recommend using now.
Especially when we’re in a time like this with a lot of change and fluctuation and really rapidly changing outlooks and needs, I would say the number one thing that you can do is – and it sounds so simple – talk to people. Whether they’re your current users or they used to be your users or you want them to be your users, there’s something so powerful about a one-on-one moderated interview. You can throw in other things; you can also get some usability feedback.
Moderated user interviews sound really obvious; however, Amanda added one tip that you might find useful: talk to the extremes.
One thing I always recommend people do is if they have something that’s live or ongoing, talk to people in the extremes of interacting with you. If you have somebody who has not slowed down their travel at all, talk to them and understand why.
What motivates them? What makes them feel safe or what doesn’t? What is their context that means they are someone who is super interactive with you or is still traveling a lot or whatever the context is? On the flipside, talk to the people on the other end of the spectrum – the people who used to fly or travel really frequently and no longer are – as well as a mixture in the middle.
One big challenge of the pandemic is that it makes doing in-person user interviews or moderated user testing more difficult. This is why leveraging remote user research can come in handy in the current situation. Here is Amanda’s tip on how to do remote moderated user interviews and testing:
Remote user research is really powerful, and it can be great for a lot of things. Especially for places like airlines, who are likely to have a user base that’s really spread out geographically, it can be really powerful to be able to talk to people not in person. It can be a little bit easier to schedule because you don’t need people to meet you at an office or go to a moderation lab.
It really allows you to get in touch with people in lots of different places – and people who maybe normally wouldn’t participate in research. Right now is an especially interesting time because there’s a lot of people who are perhaps furloughed or not as busy with work as they might usually be. I’ve actually had quite good luck the last few months interacting with people online.
Remote user research might sound like an easy thing to do, but there are some things you might need to be careful about. The first thing Amanda recommends is to record your sessions:
I absolutely suggest some sort of videoconferencing tool to do either one-on-one interviews, or you can do moderated usability tests if you can share screen or have them share yours. I really recommend recording. One thing to be extra careful about, though, is that you don’t get the full context necessarily. You can’t see their whole body. Sometimes there are some technical glitches. Like if you’re working on a prototype and they have to pull it up on their computer that’s a little bit slower or configured differently than yours, you have a little less control, especially if you’re running something where you have to share screen.
I use a tool like Zoom. There are lots of other ones where it’s a videoconference tool, and basically I can share a screen with people if I want to show them anything. I can see their face, which is important, and they can see mine, which helps it feel a little less clinical. It’s easier to build rapport. And after the fact – I also always record the sessions – I can go back and re-watch certain parts of it and see what their face was doing and read their body language as best you can. You don’t always have great views, but if somebody’s grimacing really hard or looks really frustrated, it’s pretty easy to tell. Humans in general aren’t great at hiding their emotions, no matter what they think.
The second recommendation Amanda gave concerns how you recruit people:
The other things to be careful of is that depending on how you’ve recruited people, if you don’t have exactly the right people in your sessions, that can skew your results a little bit. That’s actually true regardless of whether it’s remote or not, but with a lot of remote recruiting, it can be a little bit harder to know exactly who the people are going to be. I really recommend doing a little bit of extra screening or using a service that does a little extra screening to make sure the people are going to be able to connect, they’re going to be able to be heard and hear you, and if you’re going to have to share screen, to do a tech check and all that kind of stuff.
NOTE: Listen to the full podcast talk with Amanda and learn about all her other recommendations for remote airline UX research in detail.
The last thing I wanted to talk about with Amanda was how to embed airline UX research into agile teams and agile methodology. Because of COVID-19 and the fast-changing user behavior and context, airlines need to do faster, more frequent cycles of agile user research.
Even before the pandemic, I saw airlines that had different groups of people working in sprints – the marketing team working on campaign plans, the UX and digital product team working on development sprints, and then conversion optimization teams doing experiments.
So, how do you embed user research into this more agile and faster environment?
This is something that a lot of organizations have challenges with. I’ve seen it work really, really well, and I’ve seen lots of variations where it worked a little less well.
What works well and what I recommend is, if it’s at all possible, to have a dedicated researcher as part of an agile team – everybody is cross-functional and everybody does a little bit of everything, but if you can have somebody who’s dedicated to research within a team, that is awesome. If you can’t have that, have at least a person or a team of researchers that are dedicated to research, even if they serve multiple teams or multiple products, to have some dedicated resources.
However, there is one challenge you need to understand when doing agile UX research. This is especially relevant for airlines currently, because of limited resources and reduced staff.
A lot of times what I see is [for] agile product development teams especially, there’ll be one UX generalist person on a team who is tasked with doing both design and research work – which is better than no research work at all, but it’s problematic (1) because that’s two people’s worth of work and (2) because it’s really difficult, especially for evaluative research, to be unbiased when you’re the one who created something. It doesn’t matter how carefully you craft the questions or how much training you have; if you’re the one who designed something, it’s incredibly difficult to be unbiased.
What can airlines do about that? How can you prevent this bias?
If you really don’t have resources for that, at least pair UX generalists up and have people conduct research on other people’s designs and vice versa. So that’s the first thing, from a team setup thing. I really recommend having dedicated research, or at least shared research resources.
The other thing I’ll say is, especially in a capital ‘A’ Agile team, there’s a lot of ceremonies and scripts that go along with things, like if you’re doing two-week sprints, there’s sprint planning and there’s check-ins and then there’s the end of the sprint. There’s a lot of stuff scheduled in. I recommend doing that with research as well and having a particular cadence. It doesn’t have to be every single sprint, but as often as possible.
If you want to learn more about how to do agile airline UX research on concrete cases and work on the airline UX research framework, then check out our Airline Digital Academy.
In the Academy, you’ll be able to learn from Amanda and other experts in a small micro-community environment of fewer than 30 people. You’ll also have access to the best airline research and real case studies, and you’ll be able to interact with your mentors and peers.
No Comments