One of the best examples of deep thinking comes from the evolution of the A/B testing space. Back in the early 2000s, it was pretty difficult to gain a quantitative understanding of how your marketing efforts were working out.

In 2006, the only real A/B testing tool available was Google Website Optimizer. It looked like this:

The process was painful. You had to take your page, think about all the different sections you wanted to A/B test, and put in script tags. For each test you wanted to run, you had to create a new page.

Here are the steps Google outlines for setting up an experiment:

Name experiment and identify pages:

• Step #1 Name your experiment. For example: Sign-up Form AB Test
• Step #2 Set your original test page URL:
  http://www.mysite.com/sign-up.html
• Step #3 Set your first variation test page URL: http://www.mysite.com/sign-up-b.html
• Step #4 Set your (optional) second variation test page URL: http://www.mysite.com/sign-up-c.html
• Step #5 Set your conversion page URL: http://www.mysite.com/thank-you.html

Install and validate JavaScript tags

Original test page:

• Step #6 Add the Control Script at the top of the page.
• Step #7 Add the Tracking Script at the bottom of the page

First Variation test page:

• Step #8 Add the Tracking Script at the bottom of the page

Second variation test page:

• Step #9 Add the Tracking Script at the bottom of the page

Conversion page:

• Step #10 Add the Conversion Script

A/B experiment set-up: Preview and start experiment

• Step #11 Review the data, test the pages using the 'preview' function, and then launch your test

It was an 11-step process for a static web page. If you wanted to add different variations, or were using a website with custom CSS, you often had to spend even more time poring through the code of your site. At each step of the way, because you were altering the code of every page on your website, there was a chance that you'd introduce an error and have to start over.

Sean Ellis, who was VP of Marketing at Logmein at the time, wrote:

One way I have worked around my engineering deficiencies has been to hire the skills onto the marketing team. For example, in my last long-term VP Marketing role I hired a front-end designer/engineer to design and code landing pages and a dedicated DBA to build reports and run ad hoc queries.

Most marketers, especially at startups, simply didn't have the resources to accomplish this. Those like Sean Ellis, who were able to tap into engineering talent, could run more tests, learn faster, and grow.