Earlier, we talked about software that has revolutionized the way we organize our jobs and enable collaborative work. We focused on free software that was accessible to professionals and academics. But we discovered that both novice and experienced users have problems performing specific tasks, and these results were discovered through user testing. And today we are going to discuss a user test done by students of L’École de Design, realized by Arslan Sifaoui, Théo Geiller and Raphaël Perraud.
Test process
They started with a test of 5 seconds followed by a survey for a general understanding of the software. They then moved on to a more complete and progressive course followed by a survey to verify the understanding of the actions performed. Finally, they asked users for their impressions using open-ended questions to evaluate the quality of the product, as well as the quality of their tests.
For this preliminary test of 5 seconds (Rapid desirability test), the user will be installed in front of the computer and then be exposed for 5 seconds in front of the system so that he can express his impression. He will have to answer a series of quick questions spontaneously and describe on a blank sheet of paper the elements of the interface that he has retained. They asked for their impressions of the aesthetics of the system, the elements of the system they retained, and the functionalities they identified.
Before moving on to the pathways, they asked a sample of people to complete an introductory questionnaire on their profiles to target those who were interested in the tool and to highlight their uses/behaviors using a comparative table.
The course consisted, first of all, of editing a page where the user will have to operate as follows: write a text, put it in bold, create a checkbox, change the checkbox of place and finally transform it into a smart list. The next step will consist in creating a tree structure which will have to be shared and finally to change the workspace. This path will allow the desirability of this product and also measure tasks success and time spent. This data was compiled into a timeline, a table of numbers of clicks, and a timeline of activities.
They administered a SUS test to the users with ten questions on a scale of 1 to 5 points to determine their views on the system to assess accessibility. The results of this questionnaire will be collected to compare the scores between users.
Finally, a qualitative questionnaire with sentence completions was administered to determine the weak and strong points of the concept, to measure the desirability of the product.
Analysis results
The introductory survey showed that the users were digital natives, so they were among the potential users, but three of them answered that they preferred to take notes on paper. In the five-second test, the users had a rather good impression of the system, they could identify the navigation area, the structure of the page, the content, and the overall functionality.
With this time analysis, they were able to judge whether the user was able to complete a task over a normal (blue), fast (green), or long (red) time spent. The first competitor (Ulysse) is a regular user of Notion and uses it only for note-taking. On the other hand, the other users had more difficulties on some tasks than others and could perform simple actions, but we can observe that the novice competitors could quickly share a document.
All the tasks were completed, but we can notice that they got stuck on some steps that took them a little bit more time as we had previously seen on the time analysis, such as creating a check-box, changing the workplace, making a tree structure, sharing a document and putting in bold. This time-consuming task is the result of a fairly large number of clicks.
We can notice a correlation between the duration of the task and the number of clicks. The longer the task is going to be, the higher the number of clicks is going to be. That this is remarkable with the creation of a tree structure.
We can notice a clear difference in the user path since the fastest user (blue) serves as a reference among the other users who took time to perform certain tasks because they were exploring the software in depth.
The result of the SUS test qualified the notion software as a good software in terms of its acceptability.
Following the qualitative questions, users reported a good understanding of the system with content creation, prioritization, and work sharing. They were able to experience a tool that breaks the standards of other note software. However, they expressed frustration with the completion of certain tasks, such as the need for time to learn and get used to the system.
They summarized the results using three criteria: desirability (D), accessibility (Ac), and adoption (Ad). These indicators show the interest in the product (5 seconds test), the understanding of the functionalities (user path), and the ease of use of the product (SUS test).
Conclusion
We can notice that with the help of this user test that the users feel satisfied with the use of this product despite the problems encountered through this tool that has affected the accessibility of this product. The uncluttered aspect of the interface can make the user get lost in the very dense windows and feel frustrated when failing to perform certain tasks. The synthesis of the experience could show that the product is understood by the user even if features like workspace and tree structure are sometimes despised.
This could give me a perspective in this research, as the FH Joanneum University is appropriating a multitasking software to perform online courses as well as online assignment delivery.
I would like to explore in more detail the possibilities that Microsoft teams bring to the daily life of students and teachers. I would also like to investigate the use and usability of this product.
Bibliography:
Notion usability Test: http://www.arslansifaoui.com/files/test_utilisateur-fr.pdf
