Reducing Churn through UX: Optimizing Meeting Minutes using Data & Testing
Co-Founder & Design Lead · 2020 · 1 Eng Lead, 2 Designers, 2 Engineers
Reducing Churn through UX: Optimizing Meeting Minutes using Data & Testing
Co-Founder & Design Lead · 2020 · 1 Eng Lead, 2 Designers, 2 Engineers
Reducing Churn through UX: Optimizing Meeting Minutes using Data & Testing
Co-Founder & Design Lead · 2020 · 1 Eng Lead, 2 Designers, 2 Engineers
TL;DR
The meeting minutes feature in Buildsys - a core adoption lever - was facing significant churn.
I used data analysis and usability research to identify pain points and introduced a WYSIWYG editor for taking meeting minutes, reducing churn by 73%.
TL;DR
The meeting minutes feature in Buildsys - a core adoption lever - was facing significant churn.
I used data analysis and usability research to identify pain points and introduced a WYSIWYG editor for taking meeting minutes, reducing churn by 73%.
From a complex, click-heavy format to an intuitive WYSIWYG editor that matches how users think.
Scenario: Users can take meeting minutes in real-time and easily format them for sharing.
From a complex, click-heavy format to an intuitive WYSIWYG editor that matches how users think.
Scenario: Users can take meeting minutes in real-time and easily format them for sharing.
From a complex, click-heavy format to an intuitive WYSIWYG editor that matches how users think.
Scenario: Users can take meeting minutes in real-time and easily format them for sharing.
Context
Meetings are vital in construction, aligning stakeholders to prevent miscommunication and costly errors in a high-stakes industry.
The Meeting Minutes, part of the larger Buildsys Meetings module, allows users to:
Take Live
Minutes
Link Files
Assign &
Track Tasks
Review & Finalize
Share & Distribute
Context
Meetings are vital in construction, aligning stakeholders to prevent miscommunication and costly errors in a high-stakes industry.
The Meeting Minutes, part of the larger Buildsys Meetings module, allows users to:
Take
Minutes
Link Files
Track Tasks
Review & Finalize
Share & Distribute
Challenge
45% of users stopped using the minutes feature after 2 weeks
The meeting minutes feature in Buildsys was a key driver of user adoption, but unfortunately, it was experiencing significant churn since its rollout. This was concerning since the high churn rate of the meeting minutes feature could have a ripple effect on Buildsys as a whole.
Meetings was one of the primary selling points of Buildsys. If left unaddressed, the high churn rate in meeting minutes feature would lead to high churn rate for Buildsys.
Guys, I will get Buildsys for the meeting minutes feature alone… I love this task carry forward feature… I want to know who's tasks are delaying my projects…
Divam Kapoor · Real Estate Developer
Challenge
45% of users stopped using the minutes feature after 2 weeks
The meeting minutes feature in Buildsys was a key driver of user adoption, but unfortunately, it was experiencing significant churn since its rollout. This was concerning since the high churn rate of the meeting minutes feature could have a ripple effect on Buildsys as a whole.
Meetings was one of the primary selling points of Buildsys. If left unaddressed, the high churn rate in meeting minutes feature would lead to high churn rate for Buildsys.
Guys, I will get Buildsys for the meeting minutes feature alone… I love this task carry forward feature… I want to know who's tasks are delaying my projects…
Divam Kapoor · Real Estate Developer
Role & Team
Led design, working closely with PM, Engineering, and Sales leads, while mentoring new designers in research and analysis.
Meetings is a top priority for us, but is not being used…
Ayush · Sales Lead
Ok, let's look at the task completion rates for meeting minutes.
Yukti · Design Lead
Role & Team
Led design, working closely with PM, Engineering, and Sales leads, while mentoring new designers in research and analysis.
Meetings is a top priority for us, but is not being used…
Ayush · Sales Lead
Ok, let's look at the task completion rates for meeting minutes.
Yukti · Design Lead
Data Analysis
Out of every 100 users that start creating meeting minutes, only 27 successfully publish them.
To investigate the possible reasons for high churn, I wanted to understand how the feature is currently being used.
I worked closely with the lead engineer to add relevant logging so that we could determine the usage metrics of different actions in the minutes flow.
We noticed low completion across the board, especially in creating and assigning tasks.
Data Analysis
Out of every 100 users that start creating meeting minutes, only 27 successfully publish them.
To investigate the possible reasons for high churn, I wanted to understand how the feature is currently being used.
I worked closely with the lead engineer to add relevant logging so that we could determine the usage metrics of different actions in the minutes flow.
Usability Research
To understand user pain points, we conducted 10 in-depth one-on-one interviews & usability studies.
Armed with this insight from data, we conducted 10 one-on-one interviews and usability studies with different types of stakeholders to understand pain points.
Usability Research
To understand user pain points, we conducted 10 in-depth one-on-one interviews & usability studies.
Armed with this insight from data, we conducted 10 one-on-one interviews and usability studies with different types of stakeholders to understand pain points.
What we observed
Users struggle to take minutes live in meetings due to a click-reliant UI and formatting limitations
When taking minutes live, the users’ priority is to keep up with the pace of the meeting. The minutes at this point are often unstructured. After the meeting, the minutes are organized into formal sections and line items. At this stage, tasks are also created and assigned.
With Buildsys, the users had to first create the section heading and then the line items.
Taking live minutes was tedious & click reliant.
Old experience was too click reliant. Adding a single task required 4+ clicks. They would abandon the tool and return to taking notes on paper or MS Word.
“I expect to use this live in meetings. If I can’t do that, what’s the point?"
Deeba, Project Manager
Importing from existing minutes was time consuming and difficult.
Importing a past meeting's minutes could take up to two hours. Copy-pasting didn’t retain the formatting, so users had to do it manually for all past data.
"We like how organised Buildsys makes things, but bringing past meetings data from MS Word is very challenging for me."
Atul, Project Manager
Forced a format, not flexible.
Users wanted more flexibility with the format to comply with their existing meeting templates such as creating sections within sections, bold / underline, reordering line items etc, which were not supported.
“The template is not working for us... We've got meetings with section within a section, and this only allows one level…”
Goonmeet, Architect
What we observed
Users struggle to take minutes live in meetings due to a click-reliant UI and formatting limitations
When taking minutes live, the users’ priority is to keep up with the pace of the meeting. The minutes at this point are often unstructured. After the meeting, the minutes are organized into formal sections and line items. At this stage, tasks are also created and assigned.
With Buildsys, the users had to first create the section heading and then the line items.
Old Flow vs Desired Flow
As a result, Buildsys was not being used to take live minutes. Serving as a record keeping tool, Buildsys was an extra step in the workflow.
Users were sticking to their old way of taking minutes on MS Word and collecting feedback over email before finalizing the minutes. As a last step, they may or may not import the minutes into Buildsys for record keeping.
Some powerful features such as comments to collect feedbacks, revisions and share/distribute were not even being used as a result.
Old Flow vs Desired Flow
As a result, Buildsys was not being used to take live minutes. Serving as a record keeping tool, Buildsys was an extra step in the workflow.
Users were sticking to their old way of taking minutes on MS Word and collecting feedback over email before finalizing the minutes. As a last step, they may or may not import the minutes into Buildsys for record keeping.
Some powerful features such as comments to collect feedbacks, revisions and share/distribute were not even being used as a result.
How might we
HMW optimize the meeting minutes editor for live minute taking, matching the users’ mental model?
We mapped the users' pain points and asked HMW questions for each.
Pain Point
How might we
Taking live minutes was tedious & click reliant.
...make taking minutes less click-reliant?
Importing from existing minutes was time consuming and difficult.
...enable seamless import of past minutes while preserving formatting?
Forced a format, not flexible.
...accommodate everyone’s templates?
How might we
HMW optimize the meeting minutes editor for live minute taking, matching the users’ mental model?
We mapped the users' pain points and asked HMW questions for each.
Pain Point
How might we
Taking live minutes was tedious & click reliant.
...make taking minutes less click-reliant?
Importing from existing minutes was time consuming and difficult.
...enable seamless import of past minutes while preserving formatting?
Forced a format, not flexible.
...accommodate everyone’s templates?
Hypothesis & Success Metrics
By implementing a WYSIWYG editor for the minutes feature, we expected to improve the task completion rate and as a result, reduce the overall churn rate.
With WYSIWYG we aimed to :
Make real-time minute-taking less cumbersome
Enable seamless import of past meeting minutes
Provide more formatting flexibility
Our primary metric was task completion rate i.e. the percentage of meeting minutes started that are successfully published.
We also tracked the weekly adoption and churn rates of the meeting minutes feature.
Hypothesis & Success Metrics
By implementing a WYSIWYG editor for the minutes feature, we expected to improve the task completion rate and as a result, reduce the overall churn rate.
With WYSIWYG we aimed to :
Make real-time minute-taking less cumbersome
Enable seamless import of past meeting minutes
Provide more formatting flexibility
Our primary metric was task completion rate i.e. the percentage of meeting minutes started that are successfully published.
We also tracked the weekly adoption and churn rates of the meeting minutes feature.
Key Decisions & Tradeoffs
I partnered with engineering to make research-backed decisions that balanced UX and feasibility.
Key Decisions & Tradeoffs
I partnered with engineering to make research-backed decisions that balanced UX and feasibility.
Outcome
From a complex, click-heavy format to an intuitive WYSIWYG editor that matches how users think.
Establishing a new interaction patterns based on how user’s think.
User’s Mental Model
Old UI Assumption
New Design
Take rough notes, then organize later
Enforced structure from the start, was designed more for viewing than editing/making
Freeform input, organize later
Work fast with keyboard
Click-heavy UI
Keyboard shortcuts, custom triggers
Drag and Drop / Copy paste for import
Pasting line by line
Import past minutes with features such as drag and drop, copy paste and “copy agenda” and “carry forward”.
Optimized for live minutes
The WYSIWYG editor allows users to take rough notes with ease and organize into sections, items and tasks later.
Seamless Import
Users can now seamlessly import past minutes with features such as drag & drop, copy-paste, copy agenda or carry-forward.
Outcome
From a complex, click-heavy format to an intuitive WYSIWYG editor that matches how users think.
Establishing a new interaction patterns based on how user’s think.
Mental Model
Old UI
New UI
Take rough notes, then organize later
Enforced structure from the start
Freeform input, organize later
Work fast with keyboard
Click-heavy UI
Keyboard shortcuts, custom triggers
Copy paste for import
Pasting line by line
Import with features such as drag and drop, copy paste etc
Optimized for live minutes
The WYSIWYG editor allows users to take rough notes with ease and organize into sections, items and tasks later.
Seamless Import
Users can now seamlessly import past minutes with features such as drag & drop, copy-paste, copy agenda or carry-forward.
Outcome
From a complex, click-heavy format to an intuitive WYSIWYG editor that matches how users think.
Establishing a new interaction patterns based on how user’s think.
User’s Mental Model
Old UI Assumption
New Design
Take rough notes, then organize later
Enforced structure from the start, was designed more for viewing than editing/making
Freeform input, organize later
Work fast with keyboard
Click-heavy UI
Keyboard shortcuts, custom triggers
Drag and Drop / Copy paste for import
Pasting line by line
Import past minutes with features such as drag and drop, copy paste and “copy agenda” and “carry forward”.
Optimized for live minutes
The WYSIWYG editor allows users to take rough notes with ease and organize into sections, items and tasks later.
Seamless Import
Users can now seamlessly import past minutes with features such as drag & drop, copy-paste, copy agenda or carry-forward.
Backward Compatibility and Rollout Strategy
Rolling it out without breaking what existing users were used to required us to think about backward compatibility
Designing the new WYSIWYG editor was only half the challenge. To ensure backward compatibility and a seamless transition, we developed a compatibility and roll out plan.
Backward Compatibility
To preserve historical records and formatting, we implemented a content abstract layer that mapped legacy structured data model to the new unstructured format. This ensured uninterrupted workflows for existing users.
Legacy Model
New Format
Section
H1
Line Item
Numbered List
Note
Plain text
Task
Task
Rollout Plan
Rather than a blanket release, we chose a progressive rollout to reduce risk and gather feedback:
Phase
Target Customers
Goal
Beta
3 high-usage projects
Observe real-time behaviors, edge cases
Opt-in
All customers
Allowed teams to enable the new editor on a per-project basis
Default On
All new projects
Ensured new users land in the modern flow without confusion
Full Migration
All projects (after 3 months)
Used in-app guides + success manager outreach for training
Backward Compatibility and Rollout Strategy
Rolling it out without breaking what existing users were used to required us to think about backward compatibility
Designing the new WYSIWYG editor was only half the challenge. To ensure backward compatibility and a seamless transition, we developed a compatibility and roll out plan.
Backward Compatibility
To preserve historical records and formatting, we implemented a content abstract layer that mapped legacy structured data model to the new unstructured format. This ensured uninterrupted workflows for existing users.
Legacy Model
New Format
Section
H1
Line Item
Numbered List
Note
Plain text
Task
Task
Rollout Plan
Rather than a blanket release, we chose a progressive rollout to reduce risk and gather feedback:
Phase
Description
Beta
Controlled test on 3 high-usage projects to observe real-time behaviors and edge cases
Opt-in
Allowed teams to enable the new editor on a per-project basis
Default On
Rolled out to all new projects to prevent confusion for new users
Full Migration
Rolled out to all projects with in-app guides and support.
Impact
2.4× Increase in Task Completion and 73% Reduction in Churn
Task Completion 27% → 66%
After rollout and onboarding, the we saw a steady increase in the % of meeting minutes started that were successfully published until it settled at around 66%.
Meetings Churn 45% → 12%
We also saw a substantial increase across all meetings features such as comments and shares. Increase usage and adoption across meetings features led to a significant 73% reduction in churn rate.
Impact
2.4× Increase in Task Completion and 73% Reduction in Churn
Task Completion 27% → 66%
After rollout and onboarding, the we saw a steady increase in the % of meeting minutes started that were successfully published until it settled at around 66%.
Meetings Churn 45% → 12%
We also saw a substantial increase across all meetings features such as comments and shares. Increase usage and adoption across meetings features led to a significant 73% reduction in churn rate.
Learnings
You cannot fix what you cannot see
Data-driven design
We gained valuable insights into the importance of data-driven design, highlighting the need for robust data analysis to inform decision-making, inform the UXR process and drive product success.
Incorporating Logging
We recognized the importance of logging in understanding user behavior, identifying areas for improvement, and informing design decisions. We made a concerted effort to include logging across the board for both existing and future features.
Learnings
You cannot fix what you cannot see
Data-driven design
We gained valuable insights into the importance of data-driven design, highlighting the need for robust data analysis to inform decision-making, inform the UXR process and drive product success.
Incorporating Logging
We recognized the importance of logging in understanding user behavior, identifying areas for improvement, and informing design decisions. We made a concerted effort to include logging across the board for both existing and future features.