How To Perform A Technical SEO Audit

As an enterprise SEO, you've likely conducted one or many technical audits of websites made up of tens or hundreds of thousands of pages, possibly even millions. In fact, it's often a first, critical step when taking on a new website as it helps identify the biggest website issues and opportunities and ensure Google and other search engines are able to correctly interpret and index your site.

Anyone who has performed this task, however, also knows that the amount of information and errors you can get back is overwhelming. It's easy to lose sight of what matters most when doing a technical audit, delaying your ROI and program growth.

Where do you start? How do you know what is most important? How do you stay sane?

The simplest way to handle all the data at your fingertips is to organize it into four core areas:

1. Benchmarking: Where do you currently stand?
2. Identifying: What core index issues do you have?
3. Reviewing: What do your robot.txt and sitemap files look like?
4. Optimizing: Does your current site structure still serve you or can it be tweaked?

Benchmark your Current Organic Search KPIs

Before you start any technical audit, you need to get a baseline of your current organic search performance through tools you use daily like Google Analytics, Google Search Console, and any enterprise SEO platforms you're using like Botify.

You want to clearly benchmark how your organization's website is performing. Everything from total number of pages indexed to revenue generated by the channel should be taken into consideration.

The KPIs you should document include:

• Current keyword rankings (broken down by position)
• Current revenue (or leads)
• Current traffic (from organic search)
• Current % of pages indexed (Google Search Console/Sitemap/Botify Log File Analysis)
• Current trajectory of traffic

When looking at all this data, ask yourself:

• What pages on our site aren't getting any traffic, but should be?
• Are our main product/service pages generating good amounts of traffic and leads?
• Are there some surprise folders getting traffic that shouldn't be?
• What is a realistic growth goal after you start implementing changes?
• What percentage of the site compared to the sitemap is indexed?

Answering these questions can help identify what areas of the site you should work on first.

Benchmarking is a simple step, but one that many SEOs fail to do before performing an audit. If you can't prove the value you are bringing to the table, you could set your SEO program back months, if not years, from growing in your organization.

After you establish your baseline, it's time to get into the weeds of the website.

Identifying Indexation Errors

"If a tree falls in a forest and no one is around to hear it, does it make a sound?"

If you post quality content on your site and it isn't indexable, will anybody read it?

Your role as an SEO is to make it as easy as possible for Google to understand and discover content on your site. While there is always a chance Google could find this content from other links on the internet or even links on your own site, we don't want to make it any more difficult than it already is.

When starting an enterprise technical SEO audit, always start with indexation.

Does the site only have ONE version indexable?

First, you need to ensure there is only one version of your site that is indexable.

For example:

• http://exampledomain.com
• http://www.exampledomain.com
• https://exampledomain.com
• https://www.exampledomain.com

If more than one of these variations is not a 301 redirect to a singular version of the site, you have a very significant SEO problem on your hands. Make sure that every variation of the domain goes to a singular place. For an additional ranking boost, make sure it's an https version of the site.

When crawling the site, also make sure all of your links go to the correct version of the domain. Change any links that are a different variation of the domain to the correct version in order to get the most SEO value out of those internal links.

Check Search Console to Ensure Pages are Indexed

The next step in your audit is to compare the pages you want indexed to what Google currently has indexed. The best way to do this is through Google Search Console where you can see the number of pages you have indexed vs. how many are in your sitemap.

Look for discrepancies between how many pages you submitted in the sitemap vs. how many are indexed according to Google.

This will help you identify

• Pages that aren't indexable (on purpose or by accident)
• Pages that are poor content quality
• Entire folders that are crawled but not indexed
• Server errors
• Schema markup that is valid (breadcrumbs, FAQ, reviews, and logos)
• Sitemap errors

Google Search Console currently only allows you to download a sample size of up to 1,000 URLs that have errors, which may not be enough URLs to fix all of the problems if you manage a larger site.

Using the Botify Log Analyzer, you can find all of the pages on your site that are crawled, but not indexed, allowing you to quickly filter and identify problem areas of your site and why they haven't been indexed.

Comparing the number of indexable URLs to the number of valid URLs, you can get a clearer picture on how your site is being entered into Google's index. If your indexable URLs and valid URLs are almost a one for one, your site is in a healthy spot.

If you aren't seeing that, however, you most likely have one of two problems:

If you have more valid URLs than indexable URLs according to Google, that most likely means you have a problem with orphan URLs that you need to integrate into your site's information architecture.

If your site has more indexable pages than valid URLs, it means you most likely have crawl budget issues as not all of your indexable pages are being indexed.

Lastly, Search Console is a great place to check your schema markup to make sure Google is able to apply the structured data you intend to put on your site. We have a complete guide on structured data for more information.

Review your robots.txt and Sitemap

Once you've done a deep look at Google Search Console, you will be able to tell what areas of your site potentially have indexing issues. From there, you need to review and optimize your Sitemap and your robots.txt files.

Why?

These files are the roadmap and directive for how Google crawls your site.

Sitemaps are very simple files that serve as a list for search engines to see all the pages on your site. Each line of a sitemap includes a singular URL.

Google Search Console can tell you if you have a sitemap, how many of those pages were successfully submitted and indexed, and where there were hiccups in your sitemap. The key here is Google only wants to see 200 status code pages in the sitemap, so make sure you are removing 300 and 400 status codes when creating your sitemap.

While the sitemap serves as the roadmap for a site, the robots.txt serves as the marching orders for search engine crawlers and can have significant impact on a website's performance in search results.

Your robots.txt tells search engine bots what pages they are allowed to crawl and which ones they should ignore. You should also add your sitemap to your robots.txt.

The most important thing to remember about your robots.txt is to make sure you are only blocking folders or specific URLs you really don't want to show up in search results. If you tell search engines not to index a folder, they won't index any of those pages in that specific folder. A mistake here could turn into a disaster and have a massive impact on critical traffic to your website.

"No one but an SEO should be touching your robots.txt file."

-- Dana Tan, Under Armour

If you need help getting started with how to optimize your robots.txt file, visit Google's documentation for more information.

If you have a lot of pages that are not indexed, chances are there is something wrong with one of these files.

Optimize Your Site Structure

Site navigation and information architecture are crucial components to a quality SEO strategy.

A well-structured site navigation and intuitive information architecture are not only important for optimal user experience, but they also build relationships between pages to increase topical authority and guide search engines to the most important pages on your site. On the other hand, a confusing website structure can prevent search engines from clearly identifying which pages are the most important.

As a general rule of thumb, you don't want your page depth (distance or clicks from the homepage) to be more than five layers deep. That means building your site in a way that creates an ecosystem of internal linking and building relevancy between topics. If you're lucky enough to be a part of a new website build or site migration, make sure you get a seat at the table and are providing clear, actionable advice on the optimal website structure. This move along can create sustainable revenue for years to come.

Here are a couple of quick suggestions for how you can improve your information architecture:

• Use Botify Intelligence to quickly find broken links and analyze page depth
• Fix any 400 and 300 status code internal links to 200 status codes to get the most authority as possible from an internal link
• Build category pages on the site that help create a strong internal linking structure across topical verticals
• Identify pages you don't need anymore

With this information, you can better understand what pieces of content you need to create in order to dominate your competition in the SERPs. Ultimately, once you have a vision and goal for your site structure, you can jumpstart content creation and generate new site traffic.

Technical Audits are Never Complete

Once you've completed all the above steps, you are done with technical SEO right?

Of course not!

Once you are done doing technical triage, it's time to fully optimize your site. Botify is dedicated to helping you elevate the impact of SEO within your organizations, providing you with revenue- or conversion-driven metrics in the platform so that you can eventually skip the back-of-the-napkin math and earn that well-deserved seat at the table.

Get in touch with us today to learn more!