UX Research – Integrating Usability into “Access to Justice”
That time when we studied the usability and navigability of four state law help websites (FL, MN, MI, OH) to support our partner NJP; a civil legal aid organization that runs Washington’s website
Many of the law help websites, in each of the 50 states, have not kept up with innovations in digital media, limiting their effectiveness in closing the justice gap.
Civil legal aid organizations maintain “law help” websites to provide information about the law to low-income people without access to lawyers.
Our UW capstone team studied the usability and navigability of the redesigned “mobile first” law self-help websites from four other states.
We handed off our research findings, demonstrating the value of applying human-centered design principles to law help websites, to the organization that would redesign Washingtons.
Northwest Justice Project (NJP)
Capstone Class – Graduate School
Remote Work Only
Four UW HCDE Grad Students
In America, there is no right to counsel in civil cases. As a result, many low-income Americans “go at it alone” without legal representation in disputes where they risk losing their job, livelihood, home, children, or seeking a restraining order against an abuser.
This “justice gap” – the difference between the civil legal needs of low-income Americans and the resources available to meet those needs – has stretched into a gulf.
State courts across the country are overwhelmed with unrepresented litigants. For instance, In New York State courts in 2015 1.8 million people appeared in court without a lawyer; 98% of tenants in eviction cases were unrepresented and 95% of parents in child support cases were unrepresented in these courts in 2013. Comparable numbers can be found in courts across the United States.
*Disclaimer – Due to the importance of reforming the U.S. Justice system, the length of this portfolio project is longer than usual to provide additional context*
As the largest civil legal aid organization in Washington State, Northwest Justice Project (NJP) receives funding from Legal Services Corporation (LSC), a federal government agency that funds civil legal aid organizations in all 50 states. In 2020, NJP received a “Technology Initiative Grant” (TIG) from LSC for the redesign of the Washington Law Help (WLH) website.
Figure 1: For 20 years, the non-profit ProBono.net has provided civil legal aid organizations with standard templates and hosting for “law help” websites, but those templates are now out of date and too restrictive.
At the time, the NJP Team was in the process of developing a Request for Proposals (RFP) it would use to hire an outside contractor that would spend two years redesigning its website. That RFP would provide the governance framework for the collaborative process of redesigning the WLH website.
The goal of this Capstone Project was to provide the NJP Team with insights into the theory and practice of human-centered design that the NJP Team can use to refine and strengthen the RFP it uses to set up that governance framework.
The following research questions describe some of the central concerns that animated
and guided the research described in this report:
RQ1: Which law help website feature is most helpful for users who are low-income people with unmet legal needs?
RQ2: What wayfinding/triage tool design is best at helping lawyers communicate clearly & asynchronously with low-income people with unmet legal needs?
RQ3: What law help website feature contributes most to low-income people’s perception that their legal needs are being met?
RQ4: What law help website feature contributes most to low-income people’s chances of finding an objectively valid solution to their legal problems?
RQ5: Even if a law help website allows low-income people to quickly find objectively valid solutions to their legal problems, will that person subjectively believe their needs have been met?
Figure 2: The 2021 WLH website design. Under the TIG, the WLH Website Redesign Team (NJP Team) will hire an outside contractor to work for two years to create a more user-centric, “mobile first” version of WLH as well as a more sophisticated back-end content management system.
In collaboration with the NJP team, our capstone team created a multi-step usability study process and carried out a combination of mixed methods to answer our research questions. The mixed methods consisted of the following steps:
Reviewing background information provided by NJP Team, as well as the academic and professional HCD literature relevant to the design of self-help legal tools for low-income people.
Carrying out a heuristic evaluation of self-help legal aid websites for other states that recently got a “mobile first” redesign, resulting in the selection of the Florida Michigan, Minnesota, and Ohio law help sites for further study.
Carrying out an unmoderated usability study using Prolific.io for which participants were given a legal issue scenario and then asked to locate a specific resource on a law help website to address that issue, resulting in the selection of the Michigan Law Help website for further study.
Carrying out a moderated usability focused on the wayfinding/navigation tools in the Michigan Law Help website for which participants were given two legal issue scenarios and then asked to locate specific resources on the Michigan Law Help website to address that issue.
Analyzing and presenting the results of the literature survey, heuristic evaluation, unmoderated usability study, and moderated usability study in the Final Report.
Preliminary hypothesis #1: The wayfinding/triage tool seems most helpful (based on NJP Team analysis and recommendations, NJP background information).
Preliminary findings #1: The wayfinding/triage tool seems to be most helpful (based on results of unmoderated and moderated usability studies).
Preliminary hypothesis #2: Wayfinding/triage tool with a clear, non-technical front end (based on heuristic evaluation of other states’ redesigned law help websites).
Preliminary findings#2: Wayfinding/triage tool with a clear, non-technical front end, good
back-end integration and effective governance all help to improve communication (based
on results of unmoderated and moderated usability studies).
Preliminary hypothesis #3: Readability of information presented on a law help website will
contribute most (based on published studies).
Preliminary findings #3: Readability plus wayfinding/triage tools & back-end integration are all significant (based on results of unmoderated and moderated usability studies).
Preliminary hypothesis #4: Wayfinding/triage tools make essential contributions.
Preliminary findings #4: Wayfinding/triage tools make meaningful contributions, but a proper back-end integration was just as important.
Preliminary hypothesis #5: Without some human contact, low-income people are unlikely to experience resources delivered via a law help website to meet their legal needs.
Preliminary findings #5: Some low-income people are more capable of finding objectively reasonable solutions to their legal problems on law help websites, while others are not. The latter may feel that their legal needs have not been met without human contact confirming the validity of their results.
Figure 3: Analysis of the moderated usability study results for low income participants using law help websites was presented as a user journey map.
This study uses “mixed methods” research, i.e., research that combines elements of both quantitative and qualitative analysis:
The heuristic evaluation of redesigned law help websites from other states was carried out using qualitative research methods, i.e., collaborative evaluation of the compliance of each website with recognized usability principles.
The results of the heuristic evaluation were combined with the results of the academic and professional literature review to create the usability study kit for the unmoderated usability study.
Pilot tests were carried out for the tasks in the unmoderated study usability test kit to ensure they were feasible for participants to complete and produced relevant results.
The data generated by the unmoderated usability study participants were analyzed
statistically and the results were presented in diagrams.
The results of the analysis of the unmoderated usability study data were used to create a usability study kit for the moderated usability study, which was also run through pilot tests before it was used with the study participants.
The results of the moderated usability study were analyzed and coded, i.e., assigned descriptive words or phrases intended to capture the essence of participant comments and behaviors and those descriptive words and phrases were organized into “affinity diagrams”, i.e. arranged in groups to indicate relationships or patterns among the ideas expressed in the codes.
The research findings suggest that using iterative design processes informed by widely recognized design principles on law help websites could have a significant impact on access to justice.
Figure 4: The wireflow from the user journey map depicts how a user would find a resource on the website and how then a confusing intrusive pop up would incorrectly redirect people to the triage tool leading them away from the correct answer.
This Capstone Project used some of the tools of human-centered design to investigate what impact navigability/triage tool design decisions in law help websites have on the ability of civil legal organizations such as NJP to reduce the justice gap in America.
The results of the literature survey, heuristic evaluation, unmoderated usability study, and moderated usability study were shared in a Final Report which included:
Analysis of the data from the unmoderated usability study presented in visual form.
Analysis of the moderated usability study results presented as a user journey map for low-income people using law help websites.
Analysis of the moderated and unmoderated usability studies results presented as Website Usability General Principles.
Figure 5: Shared in our final report were navigability design suggestions for law help websites in an effort to reduce the justice gap in America.
Many organizations and individuals are both struggling to close the justice gap and navigate the tangle of incompatible solutions available today. The choice to redesign the WLH website while embracing mobile-first and user-centric design principles could further compound the inefficiency of the current fragmented patchwork of local legal aid organizations, or it could contribute to the development of national process and technology standards for making the delivery of civil legal aid more efficient.
In the former case, individuals using the redesigned WLH website would be able to find solutions to their legal problems more easily and more often, but the scale of the justice gap in America would remain unchanged. In the latter case, coordinated efforts across local legal aid organizations could contribute to the development of systemic solutions similar to those achieved for EBT distribution systems. System solutions based on emerging technologies hold out the promise of reducing the scale of the justice gap facing all low-income people in America, whether they live in Washington State or not.
Research Questions – We found it challenging to articulate straightforward research questions due to the need to balance usability considerations and legal considerations. In addition, the context of providing civil legal aid to low-income people is very complex and hard to explain to anyone new to the topic.
Process – We initially assumed we would create conventional design deliverables for Northwest Justice Project’s use. Still, as our collaboration advanced, we shifted to evangelizing for design thinking by demonstrating the value of usability testing.
Partnership – Our work with Northwest Justice Project was highly collaborative, and NJP colleagues were unfailingly gracious and committed to learning to operationalize design thinking. We hope to continue collaborating with a Directed Research Group at UW to explore this topic further.
Depth of Analysis – Initially, we found it challenging to wrap our minds around how best to support NJP’s work on developing the RFP since we would not be providing designs since they would hire an outside contractor to manage the redesign. However, we eventually made significant progress in synthesizing multiple levels of analysis by the end of the project.
The decision to simplify the legal issues considered in this study allowed us to focus on the human-computer interaction issues; however, it placed further limits on the reproducibility and generalizability of its results, but it does not invalidate our findings.
Figure 6: One year later after sharing our research report, the WLH website design has been redesigned by a contractor incorporating some of our design recommendations.
Media credit goes to Lukas, Peyton Westman, Jane K. Winn, and Ellen Reed.
Previous ProjectWebsite Development – Everledger.com
Next ProjectUI Design – Parametric Stablecoin Insurance