Perfecting Learning Experiences

The Challenge

ASBX Learn at Amazon is an internal learning platform that offers a variety of learning programs, training courses, certifications, and training events, to help Amazonians learn new skills and advance their careers.

Learn Site Prototype

Bootcamp is one of the onboarding courses offered in ASBX Learn, which is a required series for all of the Amazon new hires who fall under the tech job family. Being a part of the required onboarding path, the Bootcamp series always rank as one of the top-visited internal resources.

Aside the Bootcamp courses, ASBX Lean also offers courses and programs that help align career goals in various aspects such as technical skill growth, professional growth, or self-development.

Despite of the quality learning content being offered, we noticed the traffic to these non-Bootcamp courses and programs are dramatically lower than the Bootcamp courses.

I took the lead in identifying and analyzing the cause of this problem through user research and analysis. I was also responsible for finding a solution that will increase the number users taking the non-Bootcamp courses and our brand awareness through the redesign of our Learn catalog.

MY ROLE

Content design & strategy

Information architecture

Requirements gathering

Site development

User research

UX design

UX Writing

TEAMS

Instructional Designer

Technical Writer
PM

User Research

I planned and conducted 2 types of qualitative user research - (1) observational interviews, and (2) surveys.

The interview was focused on observing and understanding user behaviors and how users (relatively new Amazonians with less than 6 months tenure) interact with our Learn catalog. The survey was focused on learning how tenured Amazonians find learning resources and what they expect to see in the learning site.

The goal of the research was to:

identify possible root causes of the traffic drop between the Bootcamp vs. non-Bootcamp courses.
find solutions to preserve a high retention rate and induce more return rates to the Learn catalog.

The study details are as follows:

USER INTERVIEW

Observational 1:1 interviews on accessing and using technical training resources.

Participants: 20 participants within the tech job family ranging between L4-L6 with less than 6 months tenure
Methodology: Observational interview
Duration: 30 minutes per session
How: Observe how a user finds learning resources and interact with the Learn catalog.

SURVEY

Random survey at internal conferences to learn about users' preferred learning experiences. Collected more than 100 responses.

Survey Questionnaire

Root Cause Analysis

Talking to our customers and listening to their pain points, we identified the possible root causes of the traffic drop as follows:

PROBLEM

Traffic on non-Bootcamp courses are too low.

WHY

Customers leave the site after they take the Bootcamp course.

WHY

They were not aware of the Learn catalog or what's being offered.

WHY

They didn't see or didn't care to see other available learning topics.

WHY

They didn't find our learning topics relevant to their job.

WHY

Our learning topics are not easily discoverable.

IDENTIFIED CAUSES

Lack of discoverability and search optimization
Irrelevant taxonomy - too many categories
Lack of brand awareness

Key Insights

We found some key insights from our user research and root cause analysis.

From a user's perspectives,

Easy navigation: They want to easily navigate to the topics they need.
Discoverability: They want the relative learning topics presented to them first or make the topics easy to find.
Search optimization: They want the search results to be tailored to the keywords they provided.
Simplicity: They want to be able to browse other learning topics in one scroll without having to click several layers of pages.
Aesthetics: They want the layout to be visually appealing without too much texts.

From a business perspectives:

Higher traffic: We want to keep the similar traffic that we get on the Bootcamp courses on other topics as well.
Higher retention rate: We want users to stay in our site longer.
Higher return rate: Once they know about our course offerings, we want our users to consider our site as the "go to" learning site.
Higher brand awareness: We want the users to have trust in our offerings and we are the Bootcamp training owners where it has its own value already. We want to make sure that the users get the same level of learning satisfaction they get with the Bootcamp training that gets an average of 4.8 out of 5.0 satisfaction score.
Higher survey participation rate: In order to continously provide the quality learning experiences, we want our users to provide us feedback.

STRATEGY

Increase topic discoverability and search optimization.
Revamp taxonomy of the learning catalog.
Increase retention rate and brand awareness.

New Design

Overall layout & UI content

Revision V1 was all about strict categorization of the courses by Onboarding, Tech skills, Professional skills, and Tech talks. There was no theme. The courses were mixed up and listed randomly within each category. This was the version our users exited immediately after they took the Bootcamp course.

Revision V2 has improved and we introduced the "Role-based hub" concept. This revision definitely received higher retention rate and received more page visits. The courses were more tailored towards the tech job family - SDE, SDM, and TPM. We also pulled the shortcuts to the Bootcamp courses at the top to emphasize that Bootcamp branding. The users noted that they felt like the the role based hubs offered the sense of belonging.

In the Final version, I decided to remove individual cards layout per customer feedback that may be overwhelming and confusing for their similar look and feel. In this version, I introduced more personalized theme with more inviting phrases.

Instead of category titles such as "Bootcamps", "Role-based hubs", and so on, I rephrased the categories to actionable titles such as "Getting started at Amazon" so that new users know the starting point.

I didn't forget to emphasize our "Bootcamp" branding. Instead of calling out the word as a category title, I've put them as tags within the onboarding section for each role under the "Getting started at Amazon" section.

In addition, I've pulled some of the trending learning topics to the top and named it "Featured learning topics" so that the site visitors have chance to look at some of the top picked learning topics that are trending currently.

Revision V1

Revision V2

Final

Feedback Mechanism

I made a big change with the feedback mechanism as well.

Originally, the idea was to make it very simple and make it not too invasive on the page by placing on the lower right corner of the page. Using this icon-only and less invasive feedback mechanism, we received 10 feedback at the maximum per month.

Learning from this lesson, I flipped the design of the feedback widget completely and replaced the icons with words only, "Was this helpful? Yes-No". I still positioned it as a floating widget at the right bottom corner of the page for a design decision. Our users found this more inviting and the participation rate quadrupled from less than 10 per months to 40-50 feedback per month.

I took it even further and completely rephrased our feedback mechanism. On the newest revision of the Lean catalog, we really wanted to hear from our users. Then I reserved some real estate right below the hero/header section of the page for more emphasize. I also combined an icon with more favor-like phrase, "Help us improve our site". With this design decision, the feedback submission boosted and we received couple hundred responses in the first month of launching the newest catalog design.

Revised

New

Previous Version Position

Revised Version Position

New Version Position

Scoped Search

One of highly requested feedback from the user interviews and surveys was the search optimization.

The parent domain of the Learn catalog already offered the global search option in their navigation bar. However, this search often displays all of the results that includes the searched keyword as the entire domain is included in the search.

With the revised design, I added a scoped search bar right under the Learning topics section so that users have the option to search only the learning topics not the entire parent domain.

To ensure the distinction between the global search and scoped search behaviors, I phrased the placeholder texts as "Search in BuilderHub" for the global search and "Search learning topics" for the scoped search.

Global Search

New Scoped Search

Results - the success story

After the launch, we measured metrics on a monthly basis to ensure that the redesigned Learn catalog is receiving the traction from users as we have planned for this redesign project. Overall, the new Learn catalog's number of visits skyrocketed with

The details of the metrics of the new Learn catalog are as follows:

In the first three month after the launch of the new design, there were about 10k unique visitors per month, which is couple hundred times larger than the number of new hires per month. It means users are returning to the Learn catalog after taking the Bootcamp course during their onboarding.
In Q3 of 2024, the new Learn page received at least 1.5 million combined page visits, which is equivalent to the top 5% internal sites.
After completing the Bootcamp courses, the new users stayed an average of 10 more minutes on the Learn catalog site.
The user feedback indicated that 100% of the users liked the new design and layout, approximately 95% of the users liked the new scoped search experience and the new language tone of the site.
We received at least 300 feedback in the first month of launching the new design with the new feedback mechanism.