NEWS: We're launching a directory of SEO writers looking for new gigs. Check it out »
How To Pick Organic Competitors for SEO Analysis To Maximize Your On-Page Optimization Success Rate

How To Pick Organic Competitors for SEO Analysis To Maximize Your On-Page Optimization Success Rate

The thing that people love about Surfer is that it gives instant recommendations on improving on-page SEO.

But what if you introduced suggested changes and nothing happened? Or worse! What if you dropped in rankings for your money keyphrase?

There can be two reasons for that:

  1. Some other factors are holding your page back (like backlinks, poorly optimized internal structure, slow website speed, etc.)
  2. You based your work on wrong assumptions

And today, I will walk you through the process of preparing your environment for analysis in Surfer. Therefore you’ll increase your chances to rank your page with on-page optimization.

We will focus on three major things:

  1. Analyzing user intent
  2. Picking organic competitors for analysis
  3. Defining the sentiment for your page

Why Is This Important?

The right prep-work distinguishes great SEOs from mediocre ones. And since you’re here, I’m sure you belong or at least aspire to the first group.

Making content decisions based on data from search results is definitely a great idea and a good practice. The SEO glory usually lies on the first page—since those websites ranked well, chances are they’re doing something right.

But not every first page in the SERPs is the same.

Sometimes it’s a mix of e-commerce and review articles. Other times there are service pages and directories of local providers. Or maybe Wikipedia and high authority pages with bazillion links.

Should we plan page optimization based on all of them?

No, we shouldn’t and this is one of the common errors some SEOs make. Basing our analysis on the wrong competitors might result in ranking drops.

Let’s have a look at the freedomboundbusiness.com example. The author followed recommendations from Surfer and noticed he lost a few positions. Later, when he adjusted the content and gave more attention to websites he used as a benchmark, he moved from page two to page one.

Results from freedomboundbusiness.com and Surfer optimization
Source: https://www.freedomboundbusiness.com/surfer-seo-review/

As you can see, this is an essential step that will make or break your content. If you nail the pre-work process, everything that follows will be a breeze.

So how to make sure your assumptions are correct?

1. Define Search Intent

Search intent is big right now. SEO experts like Viola Eva, Josh Hardwick, and Florian Kluge agree that satisfying the search intent is THE thing that will improve your SEO performance.

Search intent defines what users expect by typing a keyword. Are they looking for information? Want to make a purchase? Find a short answer to their nurturing question?

Google’s job is to serve the most accurate results that will satisfy the users’ needs. That’s basically the core of its existence. Knowing the search intent behind a keyword is critical when you plan your next page. For example, if in one SERP we’ve got mostly how-to guides, you’ll have a hard time to rank your transactional landing page.

Let’s have a look at the “ergonomic chair” phrase in the UK:

"ergonomic chair" serp example

To figure out the search intent, go page by page, and check their content. For my keyword, I’ve got 7 transactional pages where the user intent is to make a purchase and 3 research pages where the searcher looks for more information.

Based on that, I’m pretty sure I’ll have better chances to get to the top by creating a transactional page than a catalog with ergonomic chairs images.

According to Kane Jamison from Content Harmony, we can classify search intent like this:

  • Research - searcher is looking for more information
  • Answer- searcher is looking for a quick answer
  • Transaction - searcher is looking to buy a product
  • Local - searcher is looking for a local answer
  • Visual - searcher is looking for a visual inspiration
  • Video - searcher is looking for video content
  • Fresh/news - searcher is looking for the latest news
  • Branded - searcher is looking for a specific brand
  • Mixed - searcher may have several of various intents

But there’s more than that.

Some keywords like “seo strategy” may require we understand who is using the keyword, too.

SERP for "seo strategy" in the US

I can clearly see that actionable guides are favored for this search term. However, there are no articles for a senior SEO manager who’s looking for advanced techniques or case studies. And that’s for a reason. Therefore, if we tried to rank such article we would undoubtedly struggle.

When you understand the intent behind the search, you can provide accurate content, get a better understanding of its length and phrases that should be included on this page.  

Complying with the search intent is critical. If you see two different types of user intent and you can test two types of pages, go for it. Just beware of keyword cannibalization and don’t run such experiments within one domain.

2. Pick Competitors for The Analysis

If you have defined what kind of intent your page will serve and the right angle for it, selecting competitors is a piece of cake. Simply put, you need to include pages with the same intent and exclude outliers who rank too for some reason.

This is how to pick the right competitors from the top 10:

  1. Define your content type. Even if you’re going for a “research” search intent, you need to know if you’re writing a blog post, creating a video, or a landing page.
  2. Exclude pages that serve different intent.
  3. Exclude outliers: pages that are much longer or much shorter than other ones.
  4. Exclude pages that rank because of their authority and backlink profile

Pages that are left are the organic competitors you should include in your analysis in Surfer.

Local services example:

Exclude Yelp or other directories because they are ranking for other reasons than great on-page. Exclude research articles because they target people who are not ready to buy. And you want to attract people who want to buy your services.

Then, exclude pages with 100 words—they rank due to a developed backlink profile, brand, or other factors than on-page. Now, assess your page against your direct competitors that have an amount of content similar to what you're targeting.

Blog Post vs. E-commerce Example:

If you are writing an article about something and some product pages rank next to other posts (eg. affiliate sites)—exclude them from your analysis.

In the opposite situation, exclude blog posts and long-form articles if you’re creating an E-commerce page. There are different user intents behind them, so you have to focus on comparing yourself to other pages that are the same type as yours.

Not following this rule can ruin your rankings not only when writing but when optimizing content as well. The structure and type of content we use for these types of pages are vastly different. They will warp the results when you look to create comparable content. The most frequently excluded page will be Amazon, for example—no surprise here.

Skyscraper Articles Example:

So-called skyscrapers are long-form articles that discuss a subject in-depth from different angles. They fit for keywords where people are into learning something new or during the research phase.

When you plan such an article, make sure that it fits in the niche and specific search— the user may be looking for a short answer type, if that’s the case your 10k word monster won’t fit. If you only see three similar articles in the top ten, run your analysis just based on them. It applies to the short forms as well. Long articles usually have much lower keywords density. In this case, comparing your short form with them won’t get you anywhere.

User intent and competitors determine content length, terms used, and type of communication. The fewer pages you have to exclude from the top ten, the more suitable the keyword is for your page to rank.

In a perfect scenario, you will include all the top ten pages in your analysis. Based on them, Surfer will give you precise data to act on. However, if you’ll choose your competitors wrongly, the optimization may hurt your rankings

3. Sentiment Analysis

The wrong sentiment of your content can be a hand brake for your article.

For example, try to write an article rich with positive sentiment for “smoking” keyword and get into the top 10 with it. Or top 20! I bet you’ll have a hard time, and it completely makes sense from a user’s experience.

There are some SERPs that will be very one-sided, like in the example I mentioned. When you analyze SERPs dominated by affiliates for “X review”, on the other hand, you’ll probably see pages with highly positive sentiment. It’s hard to earn any $$$ from a negative review, right?

But most phrases will have various sentiments—positive, negative, maybe some neutral ones. And Google may experiment with those SERPs like in case of “buy Instagram followers” keyword. Shoutout to Rad Paluszak for his outstanding presentation during CMSEO 2019 and this slide:

Sentiment analysis for "Buy Instagram followers"

As you can see, over 18 months, Google has changed which pages it favors in terms of its sentiment. In May 2018, it presented mostly pages with positive sentiment, for example, transactional landings offering Instagram followers to buy.

But the SERP has been changing a lot. And if you see mixed sentiment for your key phrase, chances are it will change in the next six months or so.

At this moment in time, focus on the results of your analysis in the present, look into top-ranking pages, whether they have a positive or negative sentiment. If there are 7 positive ones and 3 negative—it will be easier to outrank one of the 7 pages with positive sentiment.

84% SERPs are dominated by positive results. We analyzed 17,500 pages sentiment - read the full story.

With this step, you’ll have all you need to run your analysis with Surfer.

  • You determined search intent for your keyword
  • You excluded pages that might skew your analysis
  • You decided whether your page will have a positive or negative sentiment.

That means you’ll have a strong foundation for further work that 99% of SEOs don’t.

Now let’s put that in practice.

Example: “Software Outsourcing” for the US

Audit for “Software Outsourcing”

This will work if you’re optimizing your existing content.

Let’s go to the SERP Analyzer, type our keyword, and choose the location.

SERP Analyzer input in Surfer

When the analysis is ready, we enter the query. We’re looking for some quick wins so that we will use the Audit feature.

By default, the Audit is based on the top five pages. This will not work for my case, though, as the SERP is a mix of different types of intent and websites:

SERP view in Surfer

Just in the top five, I see:

  • Three form articles on the subject—1, 2, 3
  • Service landing page—4
  • Quora thread on what is software outsourcing—5

By now I know that that the default Audit setting is a no-no.

Let’s check the whole top ten to get a clearer view of search intent and my organic competitors.

SERP view in Surfer

Now it gets even more interesting. On positions 9 and 10 I have high authority pages: Forbes and Stack Overflow (position 9 and 10), and the homepage of some service provider (position 6). I would group this one with the service landing page for the page in position 4.

If I’m optimizing an article, I only want to compare it to other articles.

I’ll use the eye icon to exclude other pages:

How to exclude pages in Surfer

For the purpose of my work, I’ll only leave 5 pages from the top ten—websites from positions 1, 2, 3, 7, and 8.

Only then, I create the Audit for my page. If it ranks in the top fifty, find it and click the Audit icon.

"Audit" button in Surfer's SERP Analyzer

If your page is not ranking yet, use “Compare your website to competitors” module above the SERP:

"Compare your website" section in SERP Analyzer

Content Editor for “Software Outsourcing”

This option is for you if you create content from scratch.

Go to Content Editor in the app navigation, and just as previously, put your keyword and location. The input looks the same for Content Editor and SERP Analyzer.

Content Editor in navigation

Enter the query from the dashboard.

Since we already know who our competitors are, let’s pick the pages for analysis in the customization panel:

Customization panel in Content Editor: Pages to include

Once you click “Save changes” Surfer will calculate the optimal content length, structure recommendations, and a list of important phrases.

That’s all your copywriter needs to know to create an optimized content piece.

Customization panel in Content Editor: structure and phrases

Of course, you can add and remove terms from the list.

When you close the customization panel, Surfer will calculate the number of times each phrase should be used in the content. Those numbers are based on the True Density algorithm.

Important terms to use in Surfer

Conclusion

Recommendations from both SERP Analyzer and Content Editor are much more reliable when you compare oranges to oranges. Word count varies massively, so does keyword density and other similar metrics across different types. Including the wrong types of pages in your audit will result in warped data which would lead you to optimize your content incorrectly.

Follow user intent and what kind of content is favored by Google. If Google is ranking service pages in 8 out of 10 o the first page results, logic suggests your review page might really struggle to get ranked. Analyze pages that are your true competition and use the data for your advantage.

Good luck!