post

Footer About

Footer Contribute

Footer Get Involved

Footer Links

Mind the Product

Product Management

Sunday Rewind: Whose job is it anyway? The rise of Product Ops and why it deserves to be an independent function

Guest Post

Creating metrics that matter

B2B product management

Artificial Intelligence (AI) & Machine Learning (ML)

Building Product Teams

Business Strategy

Data Driven Product Management

Digital Transformation

Diversity and Inclusion

Minimum Viable Product

OKRs

Prioritisation

Product Design

Product Research

Product Development Process

Product Ethics

Product Discovery

Product Leadership

Product Management Career

Product Strategy

Stakeholder Management

🌎 <a href="https://www.mindtheproduct.com/world-product-day/" style="color:#FFFFFF;text-decoration:underline;">Happy World Product Day!</a> Be sure to celebrate product, yourself, and your peers today! 🎉

How much are product managers getting paid in 2024?

Leading product ops at Delivery Hero with Koen Dubelaar

Connecting fans to creators at Spotify (Dariusz Dziuk, Product Lead, Spotify)

“You’re not as competent as you think” 5 lessons from Ken Chin on navigating a product career (CPO, Spark Networks)

Everything wrong with the product management job market. And how to fix it

Richard runs product development at VouchedFor - a rate and review website for financial advisers, solicitors and accountants that helps people to find the best professional advice for life's most important decisions. His previous work has focused on ecommerce and personal finance products and when not working he's often found riding his bike up hills, usually very slowly.

Richard Skinner

<p>Have you ever spent months testing and optimising a site and still found yourself falling short of targets? Or ever felt that your site satisfied CRO best practice and been unsure how to get the additional conversion boost you need? What do you do next? How about throwing best practice out the window and adding 37% to revenue per form visit?</p>
<h2>Overview</h2>
<p>VouchedFor is a rate and review site for financial advisers, accountants and solicitors &#8211; think TripAdvisor for professional services. A key offering of the service is allowing consumers to contact a professional through the site. After a site redesign and several months of intensive conversion rate optimisation, our focus was on the last step in the funnel; the contact form.</p>
<p>Two months running A/B tests to optimise contact forms had had a positive impact, with several tests producing significant uplifts in conversion rate including, for example, a 29% increase in form completion rate through a button wording change.</p>
<p>However, while satisfying most conventional best practices, conversion rate of the form was short of the target the team believed was achievable, and worse, we were running out of ideas. We were left with the following options &#8211; accept defeat and shift focus to a different area of site; continue down the path we were on with increasingly niche refinements or go back to the drawing board and come up with some great new ideas.</p>
<h2>The Best Practice</h2>
<p>It’s not hard to find a list of best practices based on others’ experiments &#8211; for example, <a href="http://conversionxl.com/53-ways-to-increase-conversion-rate/">http://conversionxl.com/53-ways-to-increase-conversion-rate/</a>. These articles almost uniformly cite several optimisations that seem guaranteed to improve conversion, along with case studies where one of these changes has led to huge conversion gains, such as:</p>
<ul>
<li>Headline wording</li>
<li>Buttons wording and styling &#8211; <a href="http://www.growthgiant.com/blog/ab-testing-case-studies-june-2012/">CareLogger adds 72% to conversion rate by changing headline and buttons</a></li>
<li>Field styling</li>
<li>Using tooltips to explain fields that users are less likely to complete</li>
<li>Minimising the number of fields &#8211; <a href="https://www.imagescape.com/media/filer_public/06/94/0694c7f4-8914-4598-8871-b857fbc12737/form_case_study.pdf">Imagescape increases conversion rate 120% by shortening form from 11 to 4 fields</a></li>
<li>Using social proof &#8211; <a href="http://visualwebsiteoptimizer.com/split-testing-blog/persuasion-principles-increase-conversions/">Betfair increases conversion by 7% with social proof</a></li>
<li>Including a special offer or exclusive benefit</li>
<li>An explanation of what happens after the form is submitted</li>
<li>Displaying privacy and security notices &#8211; <a href="http://www.internetretailer.com/2011/01/05/orientalfurniturecom-drives-conversion-76-security">OrientalFurniture.com increases conversion by including a security seal</a></li>
</ul>
<p>With a form that satisfied all of these and more, we were starting to run out of inspiration for what we should do next.</p>
<h2>The Original (Control) Form</h2>
<p>Around 30 A/B tests had resulted in the following form:</p>
<p><a href="https://www.mindtheproduct.com/wp-content/uploads/2016/09/first-form-for-vouchedfor.gif"><img loading="lazy" decoding="async" class="aligncenter wp-image-9233 size-medium" src="https://www.mindtheproduct.com/wp-content/uploads/2016/09/first-form-for-vouchedfor-290x300.gif" alt="An example of a website control form" width="290" height="300" /></a></p>
<p>Every element on the form had been tested at least once, with many having been through multiple variants that led to this design, and when compared with a best-practice checklist it scored pretty well.</p>
<h2>The Insights</h2>
<p>To gain an understanding of why this wasn’t working as well as we thought possible we focused our research efforts on our users. User research had highlighted that consumers were intimidated by financial advisers and concerned about coming across as uninformed when communicating with them; field-level Google Analytics data showed that, after those who exited the form without starting it, the highest drop-off stage of the form was the free text enquiry box; Inspectlet recordings showed users having issues with this area of the form, with many starting, deleting, retyping and finally abandoning this section.</p>
<p>Previous tests which changed the ordering of the form had made little difference to the completion rates of the different fields, with the enquiry message consistently the stage with the highest drop-off rate.</p>
<p>This enquiry message was a key business requirement as part of the offering to financial advisers, so removing it from the form wasn’t an option. Any future form would have to have a text box for users to include any important information about themselves.</p>
<h2>The Hypothesis</h2>
<p>Users find it easier to answer direct questions than to fill in a free text enquiry due to uncertainty over what to say and a fear of coming across as uninformed.</p>
<h2>The Caveat</h2>
<p>More than one variable was changed at the same time so this wasn&#8217;t the perfect test. However, the insights gained from the test are still valid when considered in total.</p>
<h2>The Tests</h2>
<p>We decided to try a different direction. Instead of pursuing the concise version preached in the best practice guides, we decided to ask the user to supply much more information through a series of direct questions in the hope that users would find it easier to answer these types of questions than the enquiry box. If we could capture key information this way then it would reduce the importance of the enquiry box.</p>
<h3>Test 1</h3>
<p>A/B test of a new form (Variant A) against Control. The key differences between the Variant A and the Control were:</p>
<ul>
<li>Increase the number of fields, with up to 15 questions asked on the new form depending on the service selected</li>
<li>All new questions required either a numerical answer and/or allowed the user to choose from some given options</li>
<li>Use radio buttons or segmented controls instead of dropdowns</li>
<li>Increase form length from 1 to 5 pages</li>
</ul>
<h3>The Result</h3>
<p>As you might expect, Variant A performed worse than Control with a conversion rate 14.5% lower. However, some of the data suggested that it wasn&#8217;t worth abandoning the hypothesis just yet.</p>
<p>There were two stages on Variant A with a high drop off &#8211; the first page, where users were asked to select a service to receive advice on, and the last page, where the user was asked to complete the enquiry form.</p>
<p>More surprising were the data points where Variant A outperformed Control. Stages 2, 3 and 4, where users were asked to provide their contact information and answer the detailed questions about their finances, each had a drop off of less than 10%. This meant that users who selected a service on Variant A completed the form at a very similar rate to users who selected a service on Control, despite having been asked to provide significantly more data.</p>
<h3>The Next Insight</h3>
<p>Users struggle to identify one single area on which they need financial advice. Often they need advice in several areas or aren’t able to answer a specific area. The fear of coming across as ill-informed or of misleading an adviser by selecting the incorrect service was deterring people from choosing one service. Once again this was backed up by Inspectlet recordings showing users struggling to settle on one service.</p>
<h3>Test 2</h3>
<p>A/B test of a new form (Variant B) against Control. Variant B built on Variant A with the following amendments:</p>
<ul>
<li>Allow users to enter more than one service on the first page</li>
<li>Pre-populate the enquiry message with information gained from the questions the user had already answered, and give the user the opportunity to amend and add more information to the message.</li>
</ul>
<p><em><a href="https://www.mindtheproduct.com/wp-content/uploads/2016/09/Enquiry-form-gif.gif"><img loading="lazy" decoding="async" class="aligncenter wp-image-9202 size-medium" src="https://www.mindtheproduct.com/wp-content/uploads/2016/09/Enquiry-form-gif-300x218.gif" alt="An online enquiry form" width="300" height="218" srcset="https://www.mindtheproduct.com/wp-content/uploads/2016/09/Enquiry-form-gif-300x218.gif 300w, https://www.mindtheproduct.com/wp-content/uploads/2016/09/Enquiry-form-gif-768x558.gif 768w, https://www.mindtheproduct.com/wp-content/uploads/2016/09/Enquiry-form-gif-1024x743.gif 1024w" sizes="(max-width: 300px) 100vw, 300px" /></a></em></p>
<h2>The Result</h2>
<p>A significant improvement in first-page completion rate (65% higher than Variant A) and an improvement in the completion rate of the enquiry form (8% higher than Variant A) led to a much higher form completion rate &#8211; 15% higher than Control and 36% higher than Variant A.</p>
<p>The majority of users edited or added to the pre-populated enquiry message.</p>
<h2>Test Data<a href="https://www.mindtheproduct.com/wp-content/uploads/2016/09/image-2.png"><img loading="lazy" decoding="async" class="aligncenter wp-image-9210 size-medium" src="https://www.mindtheproduct.com/wp-content/uploads/2016/09/image-2-300x185.png" alt="graph showing test data" width="300" height="185" srcset="https://www.mindtheproduct.com/wp-content/uploads/2016/09/image-2-300x185.png 300w, https://www.mindtheproduct.com/wp-content/uploads/2016/09/image-2.png 735w" sizes="(max-width: 300px) 100vw, 300px" /></a></h2>
<h2>Overall Result</h2>
<p>A 15% increase in conversion rate and an increase in revenue per enquirer thanks to better matching of consumers with advisers due to the additional data provided by users led to an increase of 37% in revenue per form visitor. What&#8217;s more, the device category with both the biggest relative improvement and also now the highest absolute conversion rate was mobile.</p>
<h2>The Learnings</h2>
<p>Longer, more detailed forms asking the user for more information can increase conversion:</p>
<ul>
<li>Users were more likely to complete a five-stage form with multiple detailed questions than they were to answer a short one-page form.</li>
<li>Users who were able to add more than one service converted at a much higher rate</li>
</ul>
<p>Free text boxes are scary, particularly in a subject matter with high knowledge levels. Consumers find it easier to answer direct questions. When enquiry messages are required, giving the user a helping hand by suggesting what they could write helps.</p>
<p>Understanding users’ demand is as important as a great user experience and satisfying “best practice”. Gear conversion forms towards how the user is thinking, not the way you think about the questions.</p>
<h2>The So What?</h2>
<p>The internet is full of information about how to optimise conversion rates. Every major product, marketing, eCommerce and CRO blog contains articles on how to improve site conversion, many promising huge uplifts if you follow what they say. At times optimising a site can feel like a box-ticking exercise, with success ensured if you stick to the recipe prescribed elsewhere.</p>
<p>I hope our experience can help serve as a reminder that, as with all areas of product development, the right solution for your product could be completely different to the one that works for others, and the only way to find it is by putting your users at the core of whatever you’re making. Best practice articles can be a good way to get inspiration but they’re not guaranteed to be the right answer for you.</p>

Testing and Revising Contact Forms To Improve Conversion Rates: A Case Study

Richard Skinner

Read more

Comments