Making sure search engines can find your website is fundamental to securing visibility and organic rankings. While this shouldn’t be a problem for most sites, it might take some extra work to make sure your key pages are found. Solving technical SEO problems often means getting to grips with the basics of crawling and indexing and knowing where to start when it comes to identifying issues.

Are All The Pages You Want To Be Found Indexable?

There’s no point investing time in keyword research and optimisation if your most important pages are set to noindex because this means Google will exclude them from search results. In some cases, your robots.txt could also be blocking pages that you want to be indexed or the canonical tags on your site might not be set up correctly. These are all things that are worth checking out if you’ve noticed that your pages aren’t being found.

Are You Wasting Crawl Budget On Unnecessary Pages?

When you’re looking into issues surrounding indexing, you might also find that you have a lot of URLs that don’t contribute anything to your SEO strategy. Sometimes this can happen when blank or empty pages are inadvertently published on your website, or it could be something like a Thank You page for contact form submissions that you need on the site but forget to hide from Google.

Depending on your CMS, there might also be other types of low-value content like archive pages, tags and categories that are set to index by default.

Do You Have Redirect Chains?

Something else that might be eating up your crawl budget is redirect chains. 301 redirects are necessary if you’re changing or deleting a URL to make sure that traffic trying to access the old page ends up on the new page. Over time, chains can form which means multiple redirects between the link you follow and the page you end up on. This often happens as a result of adding new redirects on top of existing ones, which can make it more difficult for Google to crawl the site as well as slowing down site speed.

Do You Have Orphaned Pages?

Orphan pages are pages on your website that are not in your navigational menu or linked to from anywhere else on the site. Because search engines use links to crawl and index pages, this could mean you end up with URLs that aren’t getting found. Even if you have amazing content on the page, not linking to it makes it a lot harder for both Google and users to find it and may mean you’re missing out on opportunities to bring in traffic because these pages typically don’t rank well.

Another potential issue with orphan pages can happen when you remove a page with duplicate or outdated content from your site structure but don’t delete or redirect the URL. In this situation, the page you no longer want to be found could be stealing crawl budget from your important pages.

Some of these issues can be found by just taking a look at your sitemap.xml or digging deeper into the coverage errors on Google Search Console but there are some great tools out there like Screaming Frog that can help you get started with the basics of crawling your site.

If you’re looking for assistance with your business’ SEO or you’d like to grab one of our experts for a chat, don’t hesitate to give us a call on 01924 367105.