Data Mining – Release The Scrapers

In order to conduct the analysis we are going to need to go out and grab some data; what we are going to focus on for now are the basic elements that contribute to domain authority.

For the purposes of this analysis I am going to pick a body keyword and look at the top 10 ranking websites.

The keyword I will be using is SEO competitor analysis

Here is the page one SERP on 8/3/12:

Next we need to scrape for the data we want to analyze. Fire up your ffavorite scraping tool , and if your just getting started I recommend checking out Eppie's slidedeck: The SEO's Guide to Scraping Everything.

The data elements we are after are:

  • URL
  • Domain Authority
  • Page Authority
  • Number of indexed links
  • Number of linking root domains

Compile these in Excel for analysis, which should look something like this:

Next we will look at the types of links pointing to these websites. My favorite way to do this is to run an Open Site Explorer report through Link Detective .

This needs to be done for each ranking URL, but for the sake of this post, I am going to pick only one. Instead of choosing the big brands, Search Engine Watch and Raven Tools, I'm going to use Web Confs; www.webconfs.com/how-to-analyze-your-competitors-a...

Now we need to generate the Open Site Explorer report, which will look something like this: