If you follow Diggintravel, you’ve heard a lot lately about experimentation and how we think it’s the key to building modern airline digital products.
But to be fair (even though I’m a huge fan of experimentation), it’s not the only experimentation that matters. The key to building modern airline digital products is to combine experimentation with an agile, data-driven, customer-centric approach.
These are the core principles we built our new Airline Digital Retailing Academy on.
Now, you’ll say, this is all well and good to hear, but is there and airline that’s really doing this in practice?
Even more, how is it possible to apply this new, almost start-up mindset in the environment of a large legacy carrier?
I had these same questions, so you can imagine how intrigued I was when I first came across one of the articles about product and experimentation that Vanesa Tejada, Head of Product at LATAM Airlines, posted on her blog.
The blog is written in Spanish, and unfortunately my Spanish doesn’t go beyond “Una cerveza por favor.” 🙂
But even just looking at the titles, topics, and visuals, I was really intrigued. So, after a lot of Google-translating (and help from my wife, who speaks Spanish), I realized Vanesa is actually explaining how LATAM are applying all of the aforementioned modern principles for building airline digital products.
Listen to the new episode of the Diggintravel Podcast about building modern airline digital products via the audio player below, or read on for 5 key highlights from our talk with Vanesa:
And don’t forget to subscribe to the Diggintravel Podcast in your preferred podcast app to stay on top of airline digital and innovation trends!
If you ever read any of Vanesa’s articles or even are as lucky as I was and get a chance to talk to her, you’ll quickly realize one thing: you can see how passionate she is about learning, especially learning from customers. You can see this even by looking at her LinkedIn cover image:
Furthermore, this is how she described her role as a Head of Product at LATAM:
As a Head of Product, I’m mainly supporting the development of product managers in order to create a culture of customer centricity, data-driven, to focus on outcomes over outputs, to share with them how important it is to learn from customers continuously, how important it is to manage effectively the product and the development, and on top of this, the success of the product is to foster people collaboration because we have very important and very big professionals in our digital area. Thanks to their interactions, they are able to produce a lot of value for the company.
I was really happy to see that continuous learning and learning from customers are where everything starts for Vanesa and LATAM’s digital product teams because this is what systematic conversion rate optimization is all about, and this is what we preach each year with our Airline Digital Optimization Yearbook. It’s all about listening to your customers, learning from them, and then learning from the experiments that you run to validate your learnings.
But after my podcast talk with Vanesa, I started to think more deeply about her “learn from your customers continuously” concept. She also mentioned some other principles of agile airline digital product development that really made sense, and as I went through all of them in my mind, it dawned on me:
Let me use 5 quotes from Vanesa to break down what I believe are the key elements of the “learn from your customers continuously” framework. As you’ll see, experimentation is really embedded in all of them.
One of the key challenges when building airline digital products is to move from feature to outcome mindset. Let’s face it, most digital products are developed by engineers who are technical and love features. Furthermore, most of the time they don’t have direct access to your customers. This is how Vanesa explained this challenge and how they used experimentation to address it:
We started pushing for experimentation last year. A colleague called Alexander Roganovich was working with me and pushing this very hard because it’s not only about the practice; there was also a cultural change around experimentation. Basically, when you want to move a company from features-driven to outcomes-driven, the first thing you need to do is the lesson that you may know nothing about your customers and your opinions or ideas are just things we need to validate, if they have value for and demand from our customers.
We needed to be more humble and to accept that all ideas are worth being tested. That was something quite big because they were not used to experimenting as much as we wanted to do. We said, look, we need to learn continuously from our customers.
Right now, with COVID, a lot of behaviors have changed. You need to rediscover your customers and adapt your product to what they are going to demand in the future. If you want to change quickly, you need to know what is changing in your customer. That’s why we were putting in practice experimentation to learn and also to validate the learnings and ensure that we are sending to the delivery floor what matters, because the numbers said that it’s worth investing in that.
We’ve all seen a typical airline “big bang” project like implementing a new booking engine or a new mobile app. I’m talking about projects where you implement something for 12-15 months because you’ve already invested so much time and money that nobody dares to say, “We won’t launch this.” You have to launch it, and then if it works even worse than before, you start optimizing and going back. On the other hand, the approach based on agile customer feedback loops and doing validation and testing hypotheses in between such big projects is much better.
When I asked Vanesa about it, she mentioned one interesting thing:
We also have an issue that some teams can love the solution and not the problem. It’s very difficult for them when it’s not working to accept that maybe that’s something that we need to kill. [You need to] manage the emotional impact when a team is very connected with the product and with the solution.
So, your first step towards a modern, agile airline digital product development process should be to focus on outcomes and solutions (versus features and problems). But it doesn’t end there!
When you put the above two concepts together, you also need to define the metrics that really reflect that. So you need to define metrics that are focused on the impact on the customer, not partial metrics that only make sense for you or your team. Vanesa and her LATAM digital product team use the OKR framework often used by Google to set metrics that are focused on the customers and outcomes.
We started working with OKRs last year. When you have a good sense of your data and a common understanding around your data, I think the main challenge when you start implementing OKRs is that that’s going to move you from feature-driven to a more customer- and data-driven approach.
Something that we were also putting a lot of attention into is that when you define the OKR, to ensure that the objectives are mentioning the customer, [you should be] quite focused on the impact that you want to create. Because when you impact the customer, benefits come later.
Source: www.ntaskmanager.com
The last sentence really made a strong impression on me because, especially in the airline world, I see too many metrics where we are focused on partial results (like higher click-through rates for ads, higher conversion rates, higher ancillary attachment rates, higher yield), and not the end-to-end metrics that reflect the real customer impact.
Here is another quote by Vanesa on her approach to setting customer-centric OKRs (metrics):
Something that I like to do – and that’s a personal opinion – I like to put the monetary part as a key result, but not the first one. Because sometimes other things are going to happen before you receive the benefits, like for example, conversion and click-through rate. You can also write the key results as a funnel to finally achieve those benefits. That’s helping people to understand that the customer comes first, and when they are very focused on the value of the customer, the benefits will increase after that.
NOTE: You can learn more about how to set up proper end-to-end metrics for your experimentation and airline digital products development here >> Ronny Kohavi, VP at Airbnb talking about metrics, personalization, and experimentation for airlines.
One other challenge I wanted to talk to Vanesa about is the organizational aspect. How do you embed experimentation and an agile, customer-centric approach for all people involved in the development of airline digital products? (Especially in a large organization like LATAM, where there are almost 40 different digital product teams.)
It’s one thing to build one agile, often central experimentation team, but embedding these principles in your existing organization is another, much bigger task. Here is Vanesa’s take on this:
I believe that there are two approaches when you want to start running experimentation and growth techniques. If you have a dedicated team, that team is going to be the only one developing that mindset and will send to the delivery floor what’s worth being delivered. Inside the culture of the company, I believe that some teams could believe that’s the top team, the special team that is discovering all the time.
This is the approach of having a central team running all experiments, almost like a SWAT team. This team can be fast and is much easier to build, but the downside of it is that it can become a detached team, running experiments for others. So, instead of using experimentation to learn and optimize, it can become a finger-pointing exercise where the results are used to show failure.
Vanesa is a proponent of a decentralized approach, where experimentation and continuous learning from customers is embedded in all digital product teams:
Marty Cagan, [in] his book, INSPIRED, said, ‘What do you want? Do you want to create missionaries or mercenaries?’
That schema (one centralized team) is more related to mercenaries, but we wanted all the teams to experiment because that’s also related to our velocity. Why would I want one team doing experimentation if I can have 40?
But when you have experimentation spread across all the teams, there is something else you need to take into account: the experimentation is going to compete with the delivery. So you need to push all the teams to understand how to prioritize experiments and the things they have discovered and they must deliver to ensure that both discovery and delivery are running continuously and running in parallel.
NOTE: You can learn more about different organizational models for experimentation and digital product development in this article and interview with Stefan Thomke. He’s the author of the book Experimentation Works and an authority on the topic of innovation.
Source: Experimentation Works, Stefan Thomke
The last point made by Vanesa, about the velocity, is very important. If you can have several airline digital product teams working and experimenting in parallel, then you can implement new things much faster.
And as you’ll see in our last principle – speed really matters.
One interesting thing I saw Vanesa has written about in the past is that LATAM also measures the time it takes to run an experiment from beginning to end. So, they measure all phases of running experiments, from having an idea and defining the experiment to production testing through to validating and launching.
To me, it was very interesting that LATAM’s digital product teams measure all these different phases so they can see how long it takes for an experiment to go from beginning to end. As you’ll see, this is also connected to velocity and implementing airline digital products faster.
But first, how did they come up with this framework, and how does it work in practice?
When I arrived at LATAM, my first mission was to create a common culture to deliver value continuously. In practice, what we did was we created two flows, one for the experimentation and one for the delivery. We standardized the common flow to do experimentation and the common flow to deliver the product. We co-created the flow with different roles inside the digital area because we wanted to create a flow to deliver value, even in experimentation and in the delivery area.
We amended our tools to use that flow. People understood the flow, understood that we wanted to produce value thanks to different techniques, and as soon as we adapted our tools to do that, they started working on it, because also they were participating from the beginning on this initiative.
Measuring the time it takes to create a discovery with an experiment and then deliver the solution in practice is really interesting. Why is measuring this so important for LATAM?
We work with JIRA, and we have quite good people able to read all the information that JIRA tracks; every single movement of our items to extract our time to market – in the case of experimentation, we call it time to discover – and all the time that every item was at a single stage of that cycle, that flow. That helped us out because when you want to deliver product, time matters.
Sometimes we say the stakeholders are putting deadlines. No, there is a bigger deadline. The bigger deadline is the one that the customer outside is putting on us because if he or she doesn’t like what we are doing, it’s super easy to move to a different company that’s doing something similar.
So time matters, and it’s very important to optimize the product we are delivering, but it’s also very important to optimize our operations. We need to measure our operations to understand where we can optimize our time to market, our time to discover, to simplify a process and also to reach the market and the customer faster. That was the purpose of that initiative.
Example of how LATAM tracks experimentation KPIs; source: vanesatejada.com
Do you want to build modern airline digital products?
To do that, you’ll need two things: skills and a peer-to-peer network of like-minded digital pros to exchange ideas.
Well, good news: we’ve got you covered! You can do both in our new Airline Digital Retailing Academy.
The Academy is built on a micro-community (max 30 people in the whole group) and mentored training (you get direct access to 2 mentors and 4 great digital expert lecturers) model. In addition, airline professionals like you will work in small groups (5-8 people) on other topics based on your preferences in our deep dives.
We really believe this is the best way for you to learn and think about the new models for an airline digital experience fit for the future.
No Comments