The Ricardo product team works in 6-week production cycles with cross-functional teams realizing so-called opportunities in each cycle based on the prior defined OKRs, or business goals. In January one of the high-priority goals set was to increase buyer conversions by optimizing the process of bidding on and buying articles on the Ricardo platform.
A newly-created cross-functional Team at Ricardo
To deliver tangible results for this task, a “Purchasing” team was created with Rafael as the Product Manager. The newly assembled Purchasing team first had to find its own way around: what does the purchasing user journey include? What are the new KPIs? One of the high-priority goals set was to help visitors with a clear buying intent to become successful buyers. The team first concentrated on the alignment with the OKRs and the company’s mission and vision. These three components always set the framework for what the teams do and what decisions they make. Their objective: Improving buyer conversion by optimizing bidding and buying user stories. The purchasing team is set up along the Product Discovery Phases by Tim Herbig, which is a framework for agile teams.
PUX getting involved
Anna, UX Researcher in the PUX Team, had just taken over from her colleague as the lead PUX Researcher at Ricardo. This means that she is booked for a certain fixed percentage of her working time per week and coordinates all research support from PUX at Ricardo. She was therefore newly involved in planning sessions and the decision making processes in the collaboration of PUX and Ricardo.
“This was quite exciting for me because up until then I hadn’t been directly involved in the higher-level planning and prioritization of research projects”, she tells us in the interview. “So it was great to be the one having those conversations and making the decisions about what to do and how to do it.”
Therefore, in the first planning session, she decided to support the high-priority opportunity of the “Purchasing” team.
Defining the problem
The purchasing team went through a first ideation session which they based on previous user research. Working with a user-validated journey map, they had pinpointed user stories to work on during the production cycle. Marigabrielle, Ricardo’s UX Designer played a key role in this process. Anna joined the team after this first brainstorming session and Rafael asked for research support to help the team prioritize those identified stories. Their objective from the user research was to find out why users with the intent to buy a product on Ricardo didn't end up purchasing the article. In this question, users with “buying intent” were defined as users who (1) bookmarked products (2) asked questions on a product page (3) visited the same product page more than three times. Their suggestion was to start off with a specific area, such as bookmarked items, and conduct a survey to find out why users weren’t buying items they had bookmarked. Rafael at this moment hoped for tangible results within a short time in order to create value for the users already in this production cycle. His goal was to discover actionable insights to start ideating, creating prototypes, validating them, refining, developing, measuring, learning, and iterating.
Anna struggled with the research design: “Thinking about how to write such a survey, I couldn’t come up with questions that weren’t based on the #1 red flag for any researcher, assumptions. And thinking through the problem more, I wasn’t comfortable with the underlying assumption that only users with purchasing intent bookmarked items. ...or asked questions. ...or visited product pages multiple times. In fact, we didn’t really know much at all about the purchasing journey from the buyer’s perspective between the point where they choose Ricardo and the moment of truth when they click “Bid” or “Buy Now”.”
Therefore, the UX Researcher suggested taking a step back and validating the underlying assumptions of those questions in order to understand more about the users’ purchasing journey in general and their Jobs To Be Done. The central argument for her was: We don’t know why they bookmark items, what they think when they search for a product and why they ask questions on the articles. Her methodology to answer these questions: Exploratory user interviews.
Rafael was skeptical at first. He didn’t fully understand what to expect from foundational research like Anna was suggesting and how it would help the team to reach their production cycle objectives... And this generic user research would be delaying the production. In order for Anna to validate the assumptions prior to diving into deeper layers of the user journey and for Rafi to keep his 6-week production target, they agreed to take no longer than 2 weeks for those exploratory interviews.
Anna says this is the point where she missed the opportunity to make sure everyone was aligned on the specific goals and deliverables for the project. “Our research process at PUX involves several feedback rounds before the “real” research even starts: once on a project kick-off document, to agree on the scope, goals, and success metrics; again on the research plan, to make sure the research questions cover what the stakeholders want to know. I completely skipped the first round and dove straight into the research questions, and I didn’t take the time to properly follow up for feedback.”
The Research process
Anna decided on a method that was a combination of semi-structured interviews and usability testing. And she did it remotely (yes, even before “corona made it cool”, as she points out). For her, this decision was based on two main arguments:
- In her experience, participants open more up and give more realistic answers when they are in their own environment using their own device
- The timing question. It simply saves a lot of time
In total, she talked to six participants and asked them about their experience shopping online in general and observed them using the Ricardo platform in order to buy a product while they were thinking out loud.
Rafael and his team were in some interviews listening in. He remembers that it proved to him to be very insightful. However, the user interviews gave him the impression that Anna was focusing too much on the search and platform choice during these interviews while not providing enough tangible insights the team could benefit from during the product cycle.
The Research Results and their impact
Although as a rule in UX Research, five participants are already enough to get a broad spectrum of qualitative insights out of user interviews, Anna was a bit nervous about the seemingly small number of data points. However, “In this case, six participants provided so many insights across the board that I was completely overwhelmed and didn’t know where to start”, she points out. She analyzed the different steps of the user journey thematically. As a result, she renamed it to “user journey spiral”, because she found out the users were jumping back and forth between the stages. She tried to explain how users act and make decisions and therefore created a journey flow map form the results together with Fred, a UX Designer form the PUX team.
When Anna first presented her research findings to the team, for Rafael, the question of the actions that were to follow on these insights was again in the center. What he got from Anna was a very detailed general user story. “But now I realize that the general development of the user journey was a central aspect for me and my team to better understand the users. How do they actually search for something? How do you use filters?”, says Rafael today.
During this research project, Anna learned that it is not possible to get immediately actionable results out of foundational research and that two weeks is not enough time to conduct it properly. In the interview she questions whether she chose the right time to conduct this generic research. However, today she understands that her findings were still very valuable.
Today, Rafael has lost his skepticism completely. To him, this exploratory research provided a foundation that helps the team to understand where they need to dive deeper into concrete user problems and conduct ideation sessions. Not long after these interviews. The “Purchasing” team conducted a survey for a specific user story and learned that many users did not proceed to buy articles after saving them because they felt that the price is too high. So now they can focus on finding solutions to this user problem.
This is what Rafael is taking from this project: It is very important that even if you have very dense goals and want to create value in a short period of time, always remember to align with the company's vision and mission and keep the OKRs in mind. Secondly, you have to find out how the users behave in the journey you are working on, because if you start jumping in with assumptions, you can easily get lost. This can cost a lot more in retrospect and can be avoided with some basic research at the beginning of the project.
Anna, on the other hand, learned the value of timeless research and now has a favourite article about it. But she also understands better that it is not always the right time to conduct foundational research. She will be more pragmatic in the future, make sure she and the product teams are aligned on the specific goals of the research she is conducting, but also make sure none of her research is disposable. Timeless research is very valuable because the insights it generates can save time in future projects where similar knowledge is needed.
Because this particular project taught her so much, Anna wrote a whole blog post on her experience and her learnings. To get more of her valuable insights, read the blog post here.