Creating one math portal for every teacher and every student
Imagine Math PK-2 (Blueprint) and 3+ (Think Through Math) came to Imagine Learning as seperate products in seperate aquisitions. In order to deliver a complete supplemental math offering, the business required the products to be combined.
imagine
Math
What could go wrong?
The Process
The Process
Empathize
Define
Ideate
Prototype
Test
Interviews
System Usability Score
Group Ideation
Affinity Mapping
Whiteboard Sketch
Usability Testing
Pendo Data Analysis
Market Survey
Problem statements
Signals & Metrics
Lo-Fi Prototyping
Site Mapping
Hi-Fi Prototyping
Contextual Inquiry
Most of the interview recruits were dissatisfied users referred to us by field employees.
Check out my affinity mapping project
Reason 1
We were losing customers to competitors who had one math portal instead of two.
The 3+ portal had a system usability score of 42 or an F, making it the least usable product in our portfolio.
Reason 2
Reason 3
39% of users who logged in to the product left after viewing the dashboard.
Why
Combine?
Design Decisions
Proivded links to major reports to promote site exploration
Created one portal highlighting data origins.
Always provide an export option
Introduced side navigation to improve findability and improve SUS.
There were too many changes at once. Users don’t like it when you move their cheese.
What did we learn?
After releasing our combined portals during the back-to-school season of 2019, it quickly became apparent that we had introduced too much change. Our users had been using their respective experiences for years and now everything felt different. School admins and teachers are not as keen on adopting new technology as UX designers, product managers, and developers. While our system usability score improved, positive sentiment for our products plummeted.
When designing for a large cast of users, diversity of your sample is very very important.
During initial testing many of the users we worked with came as referals field employees. These ended up being mostly power users with the strong opinions. While the data we gathered from these users was accurate it was not representative of all users within the application. This left us with unanticipated blind spots which became painfully obvious with the release of
the new portal.
Doodles not included
Recruited a more diverse sample
In order to diversify our research sample we reached out to users directly through Pendo, targeting specific user groups. This largely circumvented the bias introduced by field employees referring. Data ended up coming in from nearly every state in the United States, instead of being dominated by the educational giants of Texas, California, and Florida. Again, this data exposed blind spots and found broader consensus than the data before.
Slowed down the changes
In order to gradually introduce changes we created a feature flagging system were users were able to view the new page or feature before sunsetting the origional. This method delivered much better data about the users experience with the page and allowed us to find our blind spots without upsetting customers. Once the new report had been out we used in product guides to ask if they were ready to transition all the way and if not why.
Reliably attracts 6% of weekly users
So we…
More Design Decisions
Provided Opportunity to switch back to old report experience.
Created learning opportunities for new features
Solicited design feedback directly from users
Diverging stacked bar chart shows the growth
Outcomes
Old
New
30%
42
64
Before
We were losing customers to competitors who had one math portal instead of two.
After
Accounting for both portals, total weekly users increased by 30% within 6 months.
The 3+ portal had a System Usability Score of 42 or an F, making it the least usable product in our portfolio.
Before
Jumped two letter grades in the system usability scale score (with lots more to improve on).
After
Before
39% of users who logged in to the product left after viewing the dashboard.
After
Weekly visits to key reports (besides the home page) nearly doubled within 6 months.
F
D
C
B
A
Old portal had the worst SUS I have ever seen
7K
13K
avg weekly visitors
Design Patterns
Page Title - Lato Bold 24px
Paragraph Text - Lato Regular 14px
List Title - Roboto Bold 14px
Secondary Text - Lato Regular #757575
Card Title - Lato Bold 20px
Colors
Type
Buttons
#3F51B5
#333333
#EDEDED
#FFFFFF
BUTTON TEXT
BUTTON TEXT
LINK TEXT
Primary
Secondary
Link or Action