Loan Document Review System
A system that simplifies the process for loan structure analysts to compare loan term sheets and loan agreement documents.
Role
Design Lead
Design Team
Just me
Year
2023
Overview
Bank of China (Hong Kong) came to us and hoped we can work together to develop a loan document review system. During the loan approval process, analysts frequently discuss details based on an approved loan terms list and also need to match them against signed loan agreement documents, which was time-consuming and easy to make mistakes.
They wanted a system to automatically detect and compare key terms, quickly identify differences, and ensure analysis accuracy.
Challenges
I was the only designer on this project which came with it’s own challenges. First of all, I didn’t have much similar experience, so I spent a lot of time researching and communicating to understand how loan analysts work and what they need. Second, the electronic and systematic way of designing these processes was also a major challenge.
In terms of technical implementation, I needed to work closely with engineers to understand the capabilities and limitations of natural language processing technology in order to design a feasible interface and interaction process.
Research
To understand the challenges faced by loan analysts, I conducted user interviews and prototype testing sessions. These helped to uncover key pain points and refine the product concept.
I found that their task is to ensure all terms in the loan term sheet are accurately included in the agreement. This process usually takes hours of manual search and comparison for a case review. And high accuracy is required to avoid costly mistakes.
Here is their current workflow:
- Collect digital versions of the term sheet and agreement.
- Manually search the term sheet and agreement to locate corresponding terms.
- Compare each term to confirm consistency.
- Record findings manually using paper or spreadsheets.
In observing analysts’ work and communicating with them, I discovered several key challenges they face:
The comparison process relies heavily on experience. Newcomers need to check 4-5 times to ensure accuracy, while experienced analysts can do it in 2-3 times. The first comparison usually has an error rate of 5%-10%. On average, each case comparison takes 3-5 hours, sometimes longer. Tracking already reviewed terms can lead to repetitive work. During the review process, analysts often need to switch between different documents. Analysts record their findings in several places, such as notes and spreadsheets.
Concepts
Seeing these challenges faced by analysts gave me a clearer idea of product design. Our product should include the following functions:
- Task-based document comparison: follows user mental models.
- Intelligent term extraction: uses NLP to automatically extract key terms and content, displayed in a visual interface.
- Clause matching: automatically identifies and matches terms in loan agreements, enhancing review efficiency.
- Review progress management: clearly shows lists of checked and pending terms.
Streamlined Workflow
First, I started discussing with engineers and thinking about how to optimize the current analyst workflow based on existing technology solutions. After many rounds of discussions and iterations, we came up with the following streamlined workflow.
- Users start by easily uploading documents through a drag-and-drop feature. The system then uses AI to automatically extract key terms.
- Analysts can review side-by-side document comparisons, with highlighted differences for quick decision-making.
- A one-click option allows for easy recording and finalizing of matches, aligning with user expectations.
- The workflow concludes with insights and reporting, offering valuable data access to enhance analysis.
Information Architecture
Next, I created the information architecture of the system to help client and our team better understand the functional classification and overall architecture of the product.
Lo-fi Wireframes
After determining the overall product structure, I began to translate it into interfaces, starting with low-fidelity wireframes. I quickly sketched out the layout and content blocks of key pages, exploring various page relationships and layouts to quickly test ideas. Working with the team, we refined a structure that became the basis for a high-fidelity prototype.
Prototypes & Usability Testing
After completing several iterations of wireframes, I proposed user testing to ensure the system met client requirements before development.
Here’s how we did it:
Deployed interactive prototypes via Maze testing platform.
Invited 5 bank loan analysts to simulate daily tasks, including:
- Uploading documents
- Creating new review tasks
- Comparing two documents
- Check the system’s automatic annotation results and manually annotate other results
These findings directly informed targeted optimizations to the document management module and document comparison features before development.
Insights from testing
Role Concept: Add Maker and Checker roles. The Maker conducts the initial review and annotation, while the Checker performs the final review to ensure greater accuracy. Key Feedback Needs: Add confirmations for upload, extraction, and comparison to reduce confusion. Process Transparency: Display the progress of term extraction and matching steps. Results Presentation: Improve layout for clarity and include task progress tracking. Batch Processing: Enable multiple file uploads and comparisons simultaneously. State Saving: Allow users to save progress and return to incomplete tasks.
Refinement
I considered the insights from the testing and made changes to the designs to improve usability.
I made some improvements:
1. User Roles & Permissions
🤔 Problem:
The client requested new roles (Maker and Checker) to support smoother workflows and ensure greater accuracy.
🎯 Solution:
Defined roles and permissions, creating diagrams to clarify responsibilities and improve the system’s workflow logic.
*Relationship diagram between roles and tasks
*Role and permission
2. Results Page Redesign
🤔 Problem:
The results layout was unclear, making it hard for users to find key information.
🎯 Solution:
- Made the most critical result tab the default view.
- Added term counts and expanded term content display.
- Introduced file states to track task progress.
*Before usability study
*After usability study
3. File Upload & Processing Page
🤔 Problem:
The upload process was unintuitive, lacked progress tracking, and preprocessing delays created frustration.
🎯 Solution:
- Redesigned the file upload process with clearer instructions and progress tracking.
- Added task details to streamline workflows.
- Introduced a preprocessing file list page to allow users to upload preprocessed files, reducing waiting time.
*Before usability study
*After usability study
UI Design
Here are some UI pages:
Sign in
User access is controlled through the client’s user authentication system.
Statistics
It provides an overview of the system’s performance over the last month.
Role: Maker
File upload and processing
Role: Maker
File comparison
Role: Checker
File comparison List
Role: Checker
Task statistics
Result
After six months of development, this automation tool achieves 95% accuracy in real-world applications. When combined with a manual review process, it ensures 100% compliance with customer requirements (there is still room for improvement in accuracy with the future introduction of AI machine learning technology). As an efficiency-enhancing tool, it has successfully met its intended goals.
Although this is not a complex system, the project’s smooth progress owes much to the deep involvement of the customer. Throughout the development process, the customer remained actively collaborative: providing timely feedback on system issues and quickly offering constructive suggestions for multiple solutions I proposed. This efficient interaction made product iteration progress very smoothly. Ultimately, the customer expressed satisfaction with the results, which gave me a great sense of achievement.
Extension Thoughts
The technology powering this system could be applied frequently in other fields, including law, insurance, real estate, and healthcare as well. Documents need to be compared at various stages of the designs in different industries, and document accuracy is extremely important in these industries.
With natural language processing improving for AI and technology that processes documents, the potential for this tool to have a wider impact is limitless. It will be exciting to go into other markets and see what we can do with this system. Such systems could change interactions with complex documents across many industries.