EMMERSION BY ROSETTA STONE

Timeline

March 2023-April 2023

My team

1 PM, 1 UX (me), 7 engineers

Overview

Within the Emmersion platform, test takers have the ability to take language assessments with an automated solution that determines their language ability within 15 minutes or less. As a result, administrators can quickly and confidently determine their perfect fit based on our proven data.

When an administrator is using test data to make decisions about candidates, they want to ensure that their test takers have completed the entire test so that they can be confident that the score accurately reflects their ability.

The redesign of the assessment serves to create a simpler interaction to avoid general test taker anxiety while encouraging longer responses from candidates in their open response answers.

The Problem

Allowing test takers to focus on the task at hand meant making some fundamental changes to the way that an Open Response question is presented. Historically, language learners have reported having anxiety when taking our test.

As a result, many test takers chose to skip a question, or have failed to even give a response at all, resulting in administrators missing a key part of understanding a test taker's language ability.

Goals

User: Decrease anxiety and encourage a longer response
Administrator: Improve confidence in score given to their applicant
Product: Give a simplified, more modern feel to the product while improving client relations

Research

Initially, we tried to match the current user flow, but added micro interactions that depending on the option, would allow for a change of color or a warning message to flash if users had not spoken for the desired amount of time. We set out to do A/B Testing with our users to test out how users responded to these designs.

Within our first couple user tests, it was evident that the most difficult part about designing this feature were the motivations behind it. Tests are innately stressful, and we quickly learned that the current design was missing the mark. We needed to go back to the drawing board, and discover where our users were experiencing friction, and decide the best course of action to take for next steps.

We continued through two more rounds of user tests with iterations based on feedback and had 3 main takeaways:

Keep text to a minimum. Non-native speakers tend to "scan" to understand instructions, sometimes not understanding everything being asked of them
Remove friction from users. Utilizing the pattern of "start" and "end" on a button instead of an icon is much simpler for international test-takers that might interpret differently.
Don't encourage bad behavior. By not inherently calling out that users should speak for 15 seconds, but rather, removing the "quit" button temporarily and greying out the "end" button, test takers tried their best to come up with a better answer.

How It Works

Anxiety No More

Meticulously reading through instructions is stressful. By creating a scannable instruction page, a test taker can understand what is being asked of them in no time.

Similarly, we made it more apparent that there is time to prepare, and that they have 30 seconds to gather their thoughts.

A Trusted Response

By encouraging a response with the ever-prominent audio waveform design, a test taker can now see their response live as it progresses through the total time, making them want to fill up the form with their most thoughtful response.

Results

The feedback we received from user testers was invaluable at understanding what the user experience needed to be.

We saw an immediate adoption of better, higher quality responses from test takers, and increased admin confidence to cross check candidate scoring from part one of the assessment as a direct result of rolling out this feature.