Research Driven Design: Learning Platform
myLearning is a learning management system (LMS) which allows employees to take mandatory compliance courses as well as professional and personal development courses. All of Comcast’s 90,000 employees are required to use this application to complete compliance and professional development courses.
The current legacy LMS design was to be updated with the a more modern aesthetic using some of the out of the box (OOTB) features. The Design team was brought on to focus on the homepage redesign.
The product team thought the OOTB features gave it an updated feel. This was true when comparing the legacy system, however, as a design team we knew we needed to do better.
My role on the Learning Project was conducting usability tests for the initially updated Learning design from the out of the box solution. I then conducted following up usability testing once we had a new homepage design. The initial findings would advise the new homepage design. The follow up testing would help us iterate a bit more and make smaller updates before launch and collect data for possible future updates.
I also worked on icon redesigns. The lack of clarity of what the icon meant was one of the many findings. I served as a support to the UI Developer, first brainstorming possible directions to go and then creating some designs before he finalized.
Legacy Design vs Initial “Out of the Box” design
Clearly, the legacy design was extremely antiquated. From the heavily text based nature to slow loading times to a poor search it has long needed a change. So when Success Factors (a SaaS) had an OOTB update, it was bound to be better. And it was. But, it did not meet today’s UX standards from basic usability, navigation, information architecture or overall visual appeal. I could see this, but I was not the user. I began usability testing.
Usability Testing: Round 1
As previously noted, this application is used by all Comcast employees. So the best users were a mix of employees unaffiliated with the product team working on the new changes. Initially the product team chose the users. I was promised impartial users with minimal to no understanding of the updated design. I created usability testing scripts testing the new OOTB design.
When I started having conversations and conducting testing, the results seemed skewed based on
Previous experience seeing the initial out of the box design. 89% of these users had seen it or were knowledgeable of it.
The majority were trainers or people who worked on the periphery of the product team and had more information about the ongoings behind the scenes than a general user.
I knew the results would not be consistent with general users, but did not want to allow for any of my own confirmation bias. I had to recruit my own users so that I could avoid any potential bias (from me or from the “expert users”) in the findings.
Expert vs General Users: Findings
I conducted usability testing with 14 participants, which I broke down between “expert” and “general” users. The expert users were associated with the product team while general users were employees with no association. One of the biggest differences between the two user groups findings were the primary reason for using the system.
Taking Mandatory Courses (Compliance related) is the primary reason
- Only 22% of Expert Users vs 100% of General Users
- Expert Users reasons include: auditing, recommending a course, reviewing reports
- General Users also search for non-mandatory courses
Among my various recommendations, ability to access mandatory compliance courses would need to be prioritized in the visual hierarchy. Our supremely talented UI Developer designed and built the new homepage design.
Note: the prioritization of Mandatory Courses as the top left active tab. After the new design was vetted and completed, I went back to a small group of users to get feedback on the new design.
Usability Testing: Round 2
In the second round of usability testing, I specifically focused on scenarios that related to what we could change. For example, in the new design the UI Developer added a carousel feature to view courses.
I wanted to ensure that the user could see the arrow directing them to view more courses. I asked the user to look for a course that I knew would require them to scroll. The findings found that 43% had trouble viewing the scroll. We needed to make the scroll more prominent. Our UI Developer made the scroll caret slightly darker, added a circle and a white background. This creating some necessary contrast.
On this project it was difficult to personally control for confirmation bias. I had many strong feelings about how to redesign the page, but needed to remove myself as much as possible, stay impartial with my usability testing approach and have it backed by user feedback.
Managing product owners
In this project the user research not only served to help with design elements such as navigation and nomenclature, but it also gave the product owners more of a lens into how general users interact with the application. A senior product team person sat in some usability testing, which showed him my approach, what the users were saying firsthand and gave him a preview to the recommendations I was going to make. I also ensured that when I recommended for changing the design, I gave some positive feedback on the OOTB design (based on the research). This way, the strong proponents of the initial OOTB design wouldn’t feel as challenged when we discussed the design overhaul.