march 2017 alexa alejandria, jesse gylling, kyle witt · “avvo.com is an online legal services...

Usability Report – Legal Forms Prototype

March 2017

Alexa Alejandria, Jesse Gylling, Kyle Witt

Usability Report – Legal Forms Prototype March 2017

Table of Contents

Table of Contents 1

Executive Summary 3 What we did: 3 Who and how we tested: 3 Summary of Key Findings and Recommendations 3

Introduction and Overview 5 About Avvo & Legal Forms 5

Description of the Study 5 Participant Selection and Recruitment 5 Participant Profile 6

Methodology 7 Overview 7 Test environment 7 Test Materials 7 Test Protocol 7 Data collection 8

Successes & Findings 10 Aesthetic Success 10 Functionality Success 10 Key Findings & Recommendations 11

Pin Tool 12

Legal Terms 14

Live Preview 16

Security 17

Beneficiaries 18

Confirmation of Completion 20

Other Opportunities 22

1


Next Steps 23 Lessons Learned 23 Further Work 23

Test Kit & Appendices Appendix A 25

Screener Survey 25

Appendix B 30 Summary of Additional Screener Survey Answers 30

Appendix C 32 Consent & Recording Release Form 32

Appendix D: 33 Moderator Script 33

Appendix E: 38 Post-Test Questionnaire (administered verbally) 38

Appendix F: 39 Note Taking Sheet 39

Appendix G: 41 Participant Scenario & Task List 41

Appendix H: 42 Prototype Screenshots 42

2


Executive Summary

What we did: Using an InVision Last Will & Testament prototype, our team designed, orchestrated, and conducted a usability study to gather evidence for whether or not the proposed revision of the current product offering would be successful or not. We defined success in this case as a functional and intuitive end-to-end experience. To accomplish this, we created a task-based study, recruited five user-representative participants, and utilized a streamlined cognitive walkthrough methodology. The goal of the study was to detect design inconsistencies/usability problem areas, identify functions that enable users to efficiently customize legal forms, and ultimately make a case for the overarching research question of whether or not the Legal Forms product is “on the right track” for the proposed build out.

Who and how we tested: Our study consisted of five participants between the ages of 18 and 54. General participant prerequisites consisted of basic computer skills, prior experience completing electronic forms, and (ideally) intention to complete a Last Will and Testament. Participants were asked to complete seven tasks on the prototype. Tasks included initiating a Last Will & Testament, assigning an executor, adding beneficiaries, and reviewing the end result. During each of the tasks, participants were interviewed using a streamlined cognitive walkthrough method to maximize the quantity of our data collection. Using OBS (monitor recording software), participants were recorded while completing the seven tasks. Their thoughts and feedback were then recorded and analyzed.

Summary of Key Findings and Recommendations Based on the participant data, the following table describes key usability concerns. Recommendations and opportunities are included with these findings: See next page.

3


Finding Recommendation

Participants did not understand the purpose of the pin feature.

A tutorial or tooltip that would explain the purpose of the feature.

Participants struggled with legal terms. Consistent definitions of legal terms via the tooltip and/or a glossary of common legal terms.

Live preview was not noticed. Move to a more visible location.

Security of login process brought concern.

Include a read-only share option and include two-step login process.

Participants struggled to add beneficiaries.

Remove dotted line around the beneficiary box.

Participants wanted more detail when adding beneficiaries.

Add more complexity the property area to allow for more detailed information of property.

Participants were unsure of what would happen after the finish button is pressed.All were unsure of whether the document was legally binding

Promote the ability to connect with a lawyer upon completion.

4


Introduction and Overview

About Avvo & Legal Forms “Avvo.com is an online legal services marketplace which provides lawyer referrals and access to a database of legal information consisting of previously answered questions, with information that includes client reviews, disciplinary actions, peer endorsements, and lawyer-submitted legal guides.” 1

During the first client meeting in January of 2017, Avvo’s Senior UX Designer, Puja Parakh, explained that the company’s current version of the Legal Forms product (www.avvo.com/legal-forms) was initially built for the purpose of the website’s search engine optimization. However, an upward trend in web traffic for Legal Forms started drawing their attention which then pushed the team to reassess the value and opportunity that the current product version presented. Looking to integrate Legal Forms into their core business model, they decided to rethink the product and user experience and began prototyping. Unsure of whether the product prototype was going in the right direction, Avvo’s UX team recruited HCDE 517 students (us) to conduct a usability study and gather evidence and data that would suggest the product’s interaction model was going in the right direction.

Description of the Study

Participant Selection and Recruitment The basic criteria for participants encompasses fundamental skill level with computers (e.g. knows how to navigate a webpage intuitively), experience doing basic internet research (e.g. have used Google or another search engine for personal research purposes), and experience in filling out long and expansive forms before (e.g. unemployment applications, FASFA applications, Annual Tax forms, etc). We requested that participants be familiar with an extensive form process because the study prompted participants to describe what they expected/predicted to see and happen during various stages of the prototype. Per Avvo, ideal participants should have

1 Company & business model description sources: https://en.wikipedia.org/wiki/Avvo

5

https://en.wikipedia.org/wiki/Avvo

http://www.avvo.com/legal-forms


a prior interest or willingness to fill out a last will and testament in addition to an interest in utilizing the web for legal documents. Participation incentives were $25 Amazon gift cards that were presented to participants as gratuity upon session completion. Participants were recruited through the use of a Google survey as a screener (see Appendix A for screener survey).

Participant Profile Ideal participant profiles included:

● Property Owners - Would consult Avvo for a myriad of services including (but not limited to) creating a Last Will & Testimony, prenuptial agreements, filing for divorce, etc.

● Long-Term Planners - Would research what goes into the process of creating a Last Will & Testimony before starting the process. Might gather all materials beforehand.

● Next of Kin - People with elderly parents of whom they are assuming the responsibility of preparation for ‘next steps’

The participant screener generated five recruits. Their details are summarized below, one participant per line.

The screener survey included several additional questions regarding participant experience with online legal services and likelihood of need to complete certain legal documents. A summary of responses to these additional screener survey questions has been made available in Appendix B.

6


Methodology

Overview Three rounds of task-oriented cognitive walkthrough sessions were conducted. Between February 22nd and 28th. Participant completion, responses, and feedback were recorded through OBS software. Participants subjective opinions were then analyzed and organized in an affinity diagram.

Test environment Tests were conducted in three locations. The University of Washington’s Foster Library study rooms, Microsoft’s offices, and a participant’s home office. These spaces were chosen to best meet the needs of the participants and ensure they would be the most comfortable when completing their Last WIll and Testament.

Test Materials The participants used a laptop computer running the Microsoft Windows 10 operating system to walk through an InVision prototype (see Appendix H) of the Last WIll and Testament form tool. Morae and OBS were used to record participant responses and feedback. Due to technical difficulties Morae was used with one participant. OBS was used for the remaining four participants.

Test Protocol For this study each participant followed a series of tasks associated with a scenario (see Appendix G). Utilizing the cognitive walkthrough method, participants were asked a series of questions at the beginning of each task. The participants were observed and recorded completing the tasks. Each participant session was designed to be completed in under an hour including a post test debrief. Each participant was given the option to stop the test at any time, if they were uncomfortable or needed a break. Before each session the participant was told that the product was being tested, not their ability to use the product. Each test session began by having the participant sign a form recording their consent to participate in the study (see Appendix C). The moderator then read from a script

7


explaining the procedure that the participant would follow and asked the participant to read the scenario and all tasks out loud (see Appendix D). The participant then read the scenario out loud. Participants were then provided tasks individually and asked to read them allowed. The moderator would then ask if the participant understood the task and helped to remedy any confusion that was encountered before beginning the cognitive walkthrough. Seven tasks were organized in sequential order in a way that a first time user would use the Avvo forms tool. After participants had completed the last task an oral post-test questionnaire was given (see Appendix E). Once the survey was completed the moderator debriefed the participant, explaining the purpose of the study and asked the participant to share anything that they found interesting about the process. After the debrief was completed participants were rewarded with a $25 Amazon gift card as gratuity.

Data collection Qualitative data was gathered from participants in the form of comments, quotes, observations, and feedback (see Appendix F for note taking sheet). Observers recorded timestamps where important occurrences took place. These timestamps were used for reference to help identify major issues and observations for each session. A summary table showing detail aspects of our data, types, method of collection, planned analysis, relevant research questions and associated tasks is provided below:

8


9


Successes & Findings

Aesthetic Success

The participant consensus regarding aesthetic was that the prototype presented an overall clear, concise, and straightforward interface design and interaction flow. With the exception of one participant disagreeing with the minimalist interface design, 4 out of 5 agreed that the prototype was generally approachable and intuitive and would use it.

Functionality Success

Since the nature of both the product and prototype largely deal with legalese (formal and technical language of legal documents that is often hard to understand), one major usability factor dealt with the initial basic understanding and familiarity of a user with these terms. One feature the Last Will & Testament prototype did well was providing definitions for terms as they were introduced in the user flow (there is more opportunity for this feature to be discussed in key Findings). To this regard, 5 out of 5 participants found the definitions extremely helpful once they were discovered. The feedback indicated that participants would have navigated outside of the website in order to find definitions for certain terms (e.g. “Executor”) which would have subsequently interrupted the interaction flow.

Another success identified was the Live Preview feature which 5 out of 5 participants indicated a liking for and agreed on its usefulness. However, there was also opportunity for improvement in this area that will be discussed in Key Findings.

Utilizing the “Next” button as an indication of successful task completion proved useful for both moderators and participants. One finding for here was although the ‘Next’ button was consistently at the bottom-right of each webpage, the ‘Finish’ button was at the top-right. 1 out of 5 participants indicated this to be slightly confusing with a very low severity. A recommendation here that isn’t discussed in the Key Findings is to move the Finish (Finish & Complete) button to the bottom right of the final document review which will also be where users naturally land after reviewing the document they’ve just created.

10


Key Findings & Recommendations The ultimate purpose of this study was to address the client’s overarching research question of whether or not the prototype was on the right track before committing to a product build out. We accomplished this by gathering evidence for whether or not the proposed revision would be successful or not by evaluating the prototype. Subsequently, we assigned the definition of success to be a functional and intuitive end-to-end user experience resulting in a customized Last Will & Testament. Other goals of the study included detecting design inconsistencies and usability problem areas and identifying functions that would enable users to efficiently customize legal forms. Table summary of findings and recommendations:

Priority Issue Finding Recommendation

Low Pin Tool Participants did not understand the purpose of the pin feature.

A tutorial or tooltip that would explain the purpose of the feature.

Low Legal Terms Participants struggled with legal terms.

Consistent definitions of legal terms via the tooltip and/or a glossary of common legal terms.

Medium Live Preview Live preview was not noticed.

Move to a more visible location.

Medium Security Security of login process brought concern.

Include a read-only share option and include two-step login process.

High Beneficiaries Participants struggled to add beneficiaries.

Remove dotted line around the beneficiary box.

High Beneficiaries Participants wanted more detail when adding beneficiaries.

Add more complexity the property area to allow for more detailed information of property.

High Completion Participants were unsure of what would happen after the finish button is pressed. All were unsure of whether the document was legally binding

Promote the ability to connect with a lawyer upon completion.

11


Pin Tool

5/5 participants had an incorrect idea of what it indicated

5/5 Guessed: term definitions, example input, other documents Participant Quote

“I would guess it shows an example of what goes in this box.” The pin feature, represented by a thumbtack icon, is positioned to the right of to each input field.The purpose of this feature is to allow users to save that particular field or page for later. Allowing users to leave the form and come back later with legal advice. When participants were prompted as to what they thought the thumbtack icon might have allowed them to do, all five participants guessed incorrectly. One participant said, “I would guess it shows an example of what goes in this box.” Other participants

12


guessed that it might provide the definition of a term, give an example of what to input, or allow you to leave the document and pull up others that you are working on. To help remedy the confusion that was experienced by participants our recommendation to include either a ‘tooltip’ or a short tutorial. A tooltip that is revealed on mouseover with brief instructions would be able to easily convey the basic used of the pin feature. A tutorial, which is more involved could be implemented to showcase for all the various features that users have at their disposal in the forms tool, while also teaching the user how to make proper use of the tool.

13


Legal Terms

4/5 participants struggled to understand legal terms used in the document

2/5 agreed they would leave the site to look up definitions Participant Quote

“I would like for the site to provide a definition of ‘Executor’.” Due to the nature of Avvo.com and the forms tool, users are presented with many legal terms. While some more advanced users may know what the terms mean already many do not. During various tasks throughout the session participants would encounter terms such as beneficiary or executor. Four of the participants tested struggled to understand the term or used it incorrectly. Two of the participants admitted that they would leave the site in order to get the information. This poses a problem and could reduce traffic to the site. To remedy this issue we recommend that there is a consistent tooltip throughout the site for legal terms. When presented for the tooltip of what executor means all participants were surprised and found the tooltip useful. They would then ask if that was present for

14


the other terms. Other suggestions for remedying this issue including a glossary or a “?” next to the word that would function similar to the tooltip.

15


Live Preview

2/5 discovered/noticed the Live Preview of their own accord 2/5 discovered it in the last third of the document process

5/5 agreed with its usefulness

Participant Quote: “This would’ve been a lot nicer if it was in a spot I could see it.”

The purpose of the live preview is to provide context for the user of where they are at in the form as well as the progress they are making. All the participants agreed the feature was useful. However only two participants noticed the feature, and those two did not notice it until in they were in the last third of the study. To make this tool more useful we recommend that it is moved to a more visible location.

16


Security

1/5 expressed concerns about the security of the login process

Participant Quote

“I expect the site to ask me to provide more personal information and in-turn I expect a more arduous login process.”

Security is an important feature when handling sensitive documents. One participant pointed out that the login process does not have two-step authentication. This worried the participant and compromised their trust in the site. While only one participant experienced the issue we believe it is one worth investigating further. To remedy this issue we recommend that a two-step authentication be added to the website to prevent malicious actors from gaining access to user data. We also recommend that the user have the ability to create read-only forms that they can share to prevent others from editing key pieces of information.

17


Beneficiaries

4/5 struggled with task

1/5 failed the task completely 1/5 wanted more constraints on property

1/5 was offended by lack of information needed Participant Quote

“I’d have to play with it to understand how to add a beneficiary.” Adding beneficiaries is one of the most important parts of creating a last will and testament. Users would be organizing everything they own in this portion of the document to be split among friends and family. Users will spend the majority of their time in the forms tool on this one page alone. As such it is imperative that this page be very user friendly. Almost all the participants struggled with this task. Participants ideas of what they were

18


being asked were varied. Most participants thought that they had to start writing the information inside the box with the dotted boundary, one thought the site was prompting for a picture to be inserted. It took participants a little while before they realized they needed to click the “add beneficiary” button and then submit the changes for them to be solidified. One participant did not successfully add a beneficiary, instead they continued without adding anyone. Adding a beneficiary was not the only issue on this page. Another problem that was brought up by participants was the lack of complexity in the add beneficiary page. When a participant gets to the point that they would input their belongings they are met with a text box. While this would solve some of the needs, participants felt like it needed more complexity. It should also be noted that one participant was offended by the lack of complexity stating that, “...this form is a representation of my life’s assets, I expect for it to ask me for more information...” In order to fix these issues with opening the add beneficiary page we recommend that the dotted line around the beneficiary information box be removed, many participants were confused by its existence and it does not provide a function. If the dotted line is kept the add a beneficiary button could be moved to the middle of the box or place on top of the box. This way the participant will see it first, increasing the chances that they try clicking on it. To resolve the issue inside the add a beneficiary we agree with what some participants suggested which was instead of a textbox that there be a table that they can put the information in. This will be more organized and allow the participant the chance to provide important auxiliary information, such as serial numbers, in a clean and orderly fashion.

19


Confirmation of Completion

5/5 didn’t know what would come next

5/5 agreed document should be reviewed by lawyer 5/5 weren’t convinced it was legally binding

Participant Quote

“That’s it?” “I want to speak to a lawyer, I don’t feel that this is legally binding.”

After participants completed reviewing the documents they were asked what they believed would happen once they hit finish. With five participants we received five different answers. Some wanted the document emailed to them, some wanted to print it off, and some wanted to save it and come look at it later. However there was one common thread. None of the participants believed that their last will and testament was legally binding. All participants claimed that they would like to get in contact with a lawyer. We recommend that when the user hits finish they are taken to page promoting the ability to get in contact with a lawyer. We believe that this would be the perfect time to

20


promote Avvo’s core business and would help put the end user’s mind at ease now that they know that their will is taken care of. Some participants noted that they would like the ability to email the document to their own lawyer upon completion.

21


Other Opportunities Throughout usability testing several comments were made that would make helpful additions to the Avvo.com Legal Forms tool. None of these issues negatively impact the end user, but would provide quality of life improvements for the product and make the process less daunting. The first opportunity is the ability to have the document autosave. Participants unanimously wanted this feature. It was mentioned that if filling out the document for real that the participants would normally go back and double check their findings. Many participants would do this before they even completed the document. One participants noted that they would like the ability to conference with legal device while working on the document. However the participant claimed that calling would cause an inconvenience stating that, “When I am watching my kids I don’t have time to talk on the phone.I would want to chat with someone if I had questions.” To remedy this the participant proposed the ability to chat with legal help. They claimed to be comfortable with someone with less legal experience than a lawyer at this phase of document completion. There was a slight oversight in the prototype that would most likely be fixed in the final version. Participants were asked to both sign in and provide their personal information. One participant mentioned that, “[It] Would be great if Avvo filed this for me when complete.” As a quality of life improvement it would be useful to save the user's personal information, this way they do not have to input it on every legal form they fill.

22


Next Steps

Lessons Learned Ultimately this study produced significant insight into what the prototype does well and where improvements need to be made. However, the path leading to the results was difficult and filled with obstacles. The limited functionality of the prototype made utilizing typical usability methods problematic. The atypical streamlined cognitive walkthrough approach was selected only after several other methods were previously decided on and subsequently abandoned upon realization that they simply weren’t feasible given our prototype. These method revisions and delay in final decision wasted much time that otherwise could have been used planning the study. To avoid this inefficiency in the future, identifying an appropriate study method will be the first priority. Avvo adopted a very hands-off approach regarding their involvement in this study. Initially we appreciated the freedom to develop our own research questions and relevant tasks. In hindsight, however, having more feedback and guidance from Avvo’s UX team would likely have enabled us to plan and conduct a more refined study with potentially more pertinent recommendations. These recommendations would help us create a better study and meet their needs more effectively. Lastly, one of the biggest issues we encountered was in recruiting participants. Our target participants were established professionals, most likely with a family. Recruiting from this pool with a measly $25 Amazon gift card meant we were actually looking for philanthropists interested in practically volunteering to take part in our study. Given this, in future studies, recruitment efforts such as screener survey deployment should be made almost immediately upon study commencement and target a much wider audience.

Further Work While we were able to collect a reasonable amount data utilizing the streamlined cognitive walkthrough method, the limited functionality of the prototype made investigating true functionality near impossible. It is likely that conducting additional usability studies using a more functional prototype that ideally incorporates the recommendations presented in this usability study will yield additional, more pertinent results.

23


Avvo has made it clear that they plan to abandon the live forms tool currently being offered through their website, replacing it with a new forms tool developed from the prototype used in this study. It is unclear if any usability studies were conducted during development of the existing forms tool. If prior research exists, a comparative review with the results of this study may prove informative. If none is available perhaps a comparative analysis between the existing forms tool and this prototype would prove useful. This study utilized the Last Will & Testament form to exemplify the process of completing any type of form with the assumption that the results would be applicable regardless of form type. However, differences in form types and resources required to complete them are vast. Testing with a variety of form types, from simple to complex, would most likely increase the accuracy of the findings as applicable to real world use.

24


Appendix A

Screener Survey

Intro: “Thank you for taking the time to answer this brief survey about your experience completing legal documents. Your participation is very much appreciated! You may be eligible to receive a $25 Amazon gift card in exchange for one hour of your time. If you are interested in participating in a one hour in-person study session please provide your contact information at the end of the survey. This survey is being conducted by master's students at the University of Washington as part of the Human Centered Design and Engineering (HCDE) program. For more information regarding the program please visit www.hcde.washington.edu/ Please direct questions and comments to [email protected]”

Questions (* = required):

How old are you? *

● 17 or under ● 18 - 24 ● 25 - 34 ● 35 - 44 ● 45 - 54 ● 55-64 ● 65-74 ● 75+

What is your gender? *

25


● I prefer not to say ● Male ● Female ● Other:

Which of the following ethnic groups do you most closely identify? *

● I prefer not to say ● White ● Hispanic or Latino ● Black or African American ● Native American or American Indian ● Asian or Pacific Islander ● Other:

What is the highest degree or level of education you have completed? *

● I prefer not to say ● Less than high school ● High school graduate (includes equivalency) ● Some college, no degree ● Associate's degree ● Bachelor's degree ● Master's degree ● Ph.D., law or medical degree

In terms of occupation, which of the following describes you ? (check all that apply) *

● I prefer not to say ● Employed for wages ● Self-employed ● Out of work and looking for work ● Out of work and not looking for work ● A homemaker ● A student

26


● Active military ● Retired ● Unable to work ● Other:

Which of the following best describes you regarding a Last Will and Testament? *

● I have one. ● I am in the process of completing one. ● I plan on completing one in the next 6 months. ● I plan on completing one sometime after the next 6 months. ● I am not planning on completing one.

Which of the following legal websites, if any, have you visited in the past year? *

● I have not visited any legal websites in the past year. ● alllaw.com ● avvo.com ● findforms.com ● freeadvice.com ● lawdepot.com ● legaltemplates.com ● legalzoom.com ● rocketlawyer.com ● Other:

If you indicated visiting a legal website in the past year, which online services did you seek?

● Speak with a lawyer ● Hire a lawyer ● Read lawyer reviews ● Obtain legal documents, forms, or contracts

27


● View legal reports, articles, or related definitions ● Other:

How likely are you to complete the following legal documents WITHOUT using legal services? * Never Possibly Probably Definitely I don't know

● Vehicle Bill of Sale ● Non-disclosure Agreement ● Durable Power of Attorney ● Last Will and Testament ● Residential Lease Agreement ● Prenuptial Agreement ● Vehicle Bill of Sale ● Non-disclosure Agreement ● Durable Power of Attorney ● Last Will and Testament ● Residential Lease Agreement ● ● Prenuptial Agreement

How confident are you in your ability to properly complete the following legal documents WITHOUT needing legal services? * Very doubtfulDoubtful Confident Very confident I don't know

● Vehicle Bill of Sale ● Non-disclosure Agreement ● Durable Power of Attorney ● Last Will and Testament ● Residential Lease Agreement ● Prenuptial Agreement ● Vehicle Bill of Sale ● Non-disclosure Agreement ● Durable Power of Attorney ● Last Will and Testament

28


● Residential Lease Agreement ● Prenuptial Agreement

29


Appendix B

Summary of Additional Screener Survey Answers

30


31


Appendix C

Consent & Recording Release Form I agree to participate in the study conducted and recorded by the Avvo Usability Research Team from the University of Washington. I understand and consent to the use and release of the recording by the Avvo Usability Research Team. I understand that the information and recording is for research purposes only and that my name and image will not be used for any other purpose. I relinquish any rights to the recording and understand the recording may be copied and used by the Avvo Usability Research Team without further permission. I understand that participation in this usability study is voluntary and I agree to immediately raise any concerns or areas of discomfort during the session with the study administrator. Please sign below to indicate that you have read and you understand the information on this form and that any questions you might have about the session have been answered. Date:________________ Please print your name: ____________________________________________________ Please sign your name: ____________________________________________________

Thank you!

We appreciate your participation.

32


Appendix D:

Moderator Script Moderator: Thank you for taking the time to participate in our study. I’m <moderator name>, I’ll be moderating today’s session. This is <observer name(s)>, <he/she/they> will be observing and taking some notes. We are master’s students in the Human Centered Design and Engineering (HCDE) program at UW conducting this study for a usability studies course on behalf of Avvo. According to Avvo, Avvo.com is the largest online marketplace connecting consumers to lawyers with over 97% of lawyers having a profile on their website. Today you will be using a limited, early prototype to of Avvo’s new online legal form tool. The purpose of this study is to identify potential design and functional concerns users may experience while using the form tool to complete a last will and testament. Our findings will be used to provide Avvo with recommendations to improve efficiency, productivity, and user satisfaction. Today we will be using what is referred to as the streamlined cognitive walkthrough method to guide your interaction with the form prototype. A scenario and a total of 8 tasks will be provided to you one at a time on a printed piece of paper. Please read each task out loud. Please wait to actually complete the task until directed to do so. Once the task is read you will be asked you a series of questions. Afterwards you will be directed to complete the task and possibly answer additional questions before proceeding to the next task. After you performing all the tasks we’ll conduct a brief interview and wrap up the session. (provide user with consent form and writing utensil) Please review and sign this consent form to indicate your willingness to participate in this study and allow us to record both audio and video throughout the session. Please be aware that these recordings will be used primarily to support our research and recommendations and may be shared with Avvo wholly or in part. Do you have any questions or concerns regarding the consent form? (address questions or concerns, collect signed consent form)

33


Ok great! Let’s get started. As previously mentioned, this is an early prototype version of the form tool with limited functionality. As such you will not able to actually type information into the forms. Keep in mind this study is not testing you or your abilities but rather your experience with the Avvo form tool.Please be as candid as possible in your responses throughout the session and let us know any time if you need a break or would like to stop the session. On the table in front of you there are several pieces of paper lying face down with a number on the back. The piece of paper marked with a zero is a scenario to provide context for your use of the form tool. Each additional piece of paper corresponds to a task. When instructed to do so, please turn the lowest numbered piece of paper over, read the scenario or task out loud, and wait for further instructions. Do you have any questions before we begin? (address questions) Tasks Review scenario Moderator: Ok, please turn the first piece of paper over, read the scenario out loud. (wait for user to complete reading) Scenario: You have come to a point in your life where you want to make sure your wealth goes to the right place after you are gone. Unsure of how to proceed you do some searching online and find Avvo.com which allows you to fill a last will and testament form for free. You find the form page and decide to begin filling it out. Moderator: Is the scenario clear to you? (provide clarification if needed) Task 1 Start state: Avvo.com last will and testament forms overview page in invision (page 1). Moderator: Ok, please turn the next piece of paper over and read the task out loud. (wait for them to read the following task) Task: You have successfully navigated to the last will and testament form overview page. Please enter the form tool. Moderator: Is the task clear to you? (provide clarification if needed) Moderator: What would you do to enter the form tool? Moderator: How would you know you are successful?

34


End state: Click get started or the image of the form Moderator: Please complete the task (wait for task to complete and begin next task) Task 2 Start state: Sign-in prompt (page 2) Moderator: Ok, please turn the next piece of paper over and read the task out loud. (wait for them to read the following task) Task: Sign into your Avvo.com user account. Moderator: Is the task clear to you? (provide clarification if needed)

Moderator: What would you do to sign into your user account? Moderator: How would you know you are successful?

Moderator: How do you feel about being prompted to sign in before starting the form? Moderator: Please complete the task (wait for task to complete and begin next task) End state: Sign in button Task 3 Start state: Last will and testament form. Moderator: Ok, please turn the next piece of paper over and read the task out loud. (wait for them to read the following task) Task Begin the form completion process for your own will. Moderator: Is the task clear to you? (provide clarification if needed)

Moderator: What would you do to begin the complete this task? Moderator: How would you know you are successful? Moderator: Please complete the task (wait for task to complete and begin next task) End state: Click the next button Task 4 Start state: Personal information page Moderator: Ok, please turn the next piece of paper over and read the task out loud. (wait for them to read the following task) Task: Add yourself as the testator.

Moderator: Is the task clear to you?

35


(provide clarification if needed)

Moderator: Do you know what a testator is? Moderator: What would you do to add yourself as the testator? Moderator: How would you know you are successful?

Moderator: Please complete the task (wait for task to complete and begin next task) Moderator: Did the testator information prompt adequately allow you to input all the required information? Moderator: Is there any other information that should have been included? Moderator: Was the tool successful in clarifying what a testator is? Moderator: How could it be made more clear what an testator is? End state: Click the next button Task 5 Start state: Executor information page Moderator: Ok, please turn the next piece of paper over and read the task out loud. (wait for them to read the following task) Task: Add an executor to the form. Moderator: Is the task clear to you? (provide clarification if needed)

Moderator: Do you know what an executor is? Moderator: What would you do to add an executor to the form?

Moderator: How do you know you would be successful? Moderator: Please complete the task (wait for task to complete and begin next task) Moderator: Did the executor information prompt adequately allow you to input all the required information? Moderator: Is there any other information that should have been included? Moderator: Was the tool successful in clarifying what an executor is? Moderator: How could it be made more clear what an executor is? End state: Click the next button Task 6 Start state: Beneficiaries page Moderator: Ok, please turn the next piece of paper over and read the task out loud. (wait for them to read the following task)

36


Task: Add a beneficiary to your form. Moderator: Is the task clear to you? (provide clarification if needed)

Moderator: Do you know what a beneficiary is? Moderator: What would you do to add a beneficiary to your form? Moderator: How would you know that you are successful? Moderator: Please complete the task (wait for task to complete and begin next task) Moderator: Did the beneficiary information prompt adequately allow you to input all the required information? Moderator: Is there any other information that should have been included? Moderator: Was the tool successful in clarifying what a beneficiary is? Moderator: How could it be made more clear what a beneficiary is? End state: Beneficiaries added, next button is clicked Task 7 Start state: Last Will and Testament Review Moderator: Ok, please turn the next piece of paper over and read the task out loud. (wait for them to read the following task) Task: Review the final document.

Moderator: Is the task clear to you? (provide clarification if needed)

Moderator: What would you do to review the final document? Moderator: How would you know that you are successful?

Moderator: Please complete the task (wait for task to complete and begin next task)

Moderator: Now that your last will and testament is complete what would be your next step? End state: Click the finish button. Task 8: Complete Post Test Interview Moderator: Thank you for completing the usability test, we just have a few more questions for you and then we are all done. (start post-test interview) END MODERATOR SCRIPT

37


Appendix E:

Post-Test Questionnaire (administered verbally)

1. Is there anything that stood out that worked well? 2. Is there something missing that could improve the product? 3. Was there anything you liked about the product? 4. Was there anything you disliked about the product? 5. What was your opinion of the live preview feature? 6. Did the live preview feature provide context on what you were filling out? 7. What is a way that the live preview feature could be more useful? 8. Would you have created a profile in order to save the form and come back? 9. When completing this form would it help to consult with a lawyer? 10. If yes how would you want to be in contact with that lawyer? 11.Would you consider using online legal services in the future? 12.How have you ever complete your taxes online? 13. If yes, did you do them all at once or over time? 14.How would you actually use this application? Gather info upfront? save and

return, etc. 15.Any other comments or suggestions about your experience?

38


Appendix F:

Note Taking Sheet Task 1: You have successfully navigated to the last will and testament form overview page. Please enter the form tool.

1. Was the participant able to get to the last will and testament form? Y / N 2. Is the task clear? 3. How would the user know they are successful? 4. How did the user struggle to access the form?

Comments and other notes: Task 2: Sign into your Avvo.com user account.

1. Was the participant able to sign into Avvo.com? Y / N 2. Is the task clear? 3. How would the user know they are successful? 4. How did the user struggle to sign in?

Comments and other notes: Task 3: Begin the form completion process for your own will.

1. Was the participant able to answer who the will and testament would be for? Y / N

2. Is the task clear? 3. How would the user know they are successful? 4. How did the user struggle to answer this question?

Comments and other notes: Task 4: Add yourself as the testator.

1. Was the participant able to input personal information for the last will and testament form? Y / N

2. Is the task clear? 3. How would the user know they are successful? 4. How did the user struggle to input their information? 5. Was the live update useful for participants? Y / N

39


6. Did the live update provide useful context and feedback as to what the participant was completing? Y / N

7. Would the pinning tool be useful? Y / N Comments and other notes:

Task 5: Add an executor to the form.

1. Is the task clear? 2. How would the user know they are successful? 3. What did the participant think was being asked of them? 4. What suggestions did the participant have? 5. Was the participant able to input executor information? Y / N 6. How did the user struggle to input executor information? 7. Was the live update useful for participants? Y / N 8. Did the live update provide useful context and feedback as to what the participant

was completing? Y / N 9. Was the participant able to understand the task they were completing? Y / N 10.What could be done to help users when they do not understand the task they are

being asked to complete? 11.Would the pinning tool be useful in this situation? Y / N

Comments and other notes: Task 6: Add a beneficiary to your form.

1. Was the user able to add beneficiaries? Y / N 2. How did the user struggle to add beneficiaries? 3. Was the participant able to understand the task they were completing? Y / N 4. What could be done to help the user when they do not understand the task they

are being asked to complete? 5. Would the pinning tool be useful in this situation? Y / N

Comments and other notes: Task 7: Review the document.

1. Was the user able to finish the form? Y / N Comments and other notes:

Post-Test Questionnaire Responses: END NOTE TAKING SHEET

40


Appendix G:

Participant Scenario & Task List

Scenario: You have come to a point in your life where you want to make sure your wealth goes to the right place after you are gone. Unsure of how to proceed you do some searching online and find Avvo.com which allows you to fill a last will and testament form for free. You find the form page and decide to begin filling it out.

Task 1: You have successfully navigated to the last will and testament form overview page. Please enter the form tool.

Task 2: Sign into your Avvo.com user account.

Task 3: Begin the form completion process for your own will.

Task 4: Add yourself as the testator.

Task 5: Add an executor to the form.

Task 6: Add a beneficiary to your form.

Task 7: Review the document.

41


Appendix H:

Prototype Screenshots https://avvo.invisionapp.com/share/JN8Z8QQA2

Thumbnail Overview

42

https://avvo.invisionapp.com/share/JN8Z8QQA2


Form Page

Doc Preview

43


Sign In

Will 1 - For Whom

44


Will 2 - Testator

Will 2.5a - Executor with Live Preview

45


Will 3a - Executor Tooltip

Will 3b - Executor Alternate Tooltip

46


Will 3d - Alternate Executor Pinned

Will 4a - Beneficiaries

47


Will 4b - Beneficiary Modal

Will 4c - Beneficiary 2

48


Will 5 - Review Document

49

march 2017 alexa alejandria, jesse gylling, kyle witt · “avvo.com is an online legal services...

Documents