How Do You Know Your Site is Properly Crawled by Search Engines?
Being in this post shows that you definitely have been trying to find an answer to some SEO related questions. Well, we know that Search engine optimization (SEO) can be tough to get your head around, especially if you’ve just launched a new business.
Some of the most common beginners’ SEO questions relate to how to ensure that Google and other search engines are aware your site exists.
In this article, I’ll discuss these basic steps in SEO and of course look into the question “Do you have to submit a URL to Google?”.
Do you have to submit a URL to Google?
As a general rule, if your site is designed with SEO considerations in mind, then Google and other search engines will be able to find and crawl new pages so they can be included in search results when appropriate. You do not have to submit a URL to Google for it to be included in its search results.
But if you don’t necessarily have to submit your links to Google, why does the thought that you should exist? There are a few reasons.
First, in the early days of SEO you did have to submit a new site for a search engine to know about it. Kind of like how you used to have to submit your business details to the Yellow Pages to be featured in that.
These days the whole process is automated and search engines actively crawl the web to identify and understand new websites and webpages. There is still a way you can give search engines a helping hand to identify and understand your website and its pages, but it’s not required that you do so. We’ll talk about this later.
Secondly, a public URL submission tool was available for Google until fairly recently. And because of that, some people assumed that the tool was the only way to get a page noticed by Google.
Thirdly, some people assume that if they can’t find their site or page on Google, it’s because they haven’t submitted it to Google. However, the real reason is most likely to be that the site/page is included in Google, it just ranks very poorly so it looks like it isn’t included in Google.
Finally, it is possible that a page or website doesn’t appear in Google at. But that’s not because you haven’t submitted it. Let’s look at how you can identify if your pages are ranking in Google and what you can do if they’re not.
How can I tell if my website/webpage is ranking in Google?
The best way to tell if your website is ranking in Google is by using Google Search Console (GSC). GSC is great because it allows you to learn and understand what Google knows about your website.
That means you can identify SEO issues and fix them to help the chances of your site ranking well in Google.
If you’ve already got GSC setup, you’ll be able to check the “Index coverage page” to see what pages are and aren’t being included in Google’s index.
Unfortunately, it will take a little while for GSC to gather information about your site so you won’t be able to instantly see if Google know about your website/pages.
However, there is a quick and easy (although less accurate) technique that you can use to check if Google knows about your site. Simply go to Google and perform the following search: Site:example.com (replace example.com with the name of your website).
You should see something like this:
If you don’t see any results at all and you’re sure you’ve enter the query correctly, then Google doesn’t know about your website.
If you’re not seeing all your pages, then it may be that Google hasn’t discovered some of them. However, just because a page doesn’t appear in a “site:” search that doesn’t mean Google hasn’t discovered it, as the search isn’t 100% accurate.
You’ll still want to set up GSC so you can get a more accurate picture, but this quick shortcut is a good way to remove any worries that Google doesn’t know about your site at all.
Finally in this section, if you’re serious about understanding your search engine rankings you may want to invest in a rank tracking tool such as SEM Rush, which can give you a better idea of what search terms your pages are ranking for, and where they appear in Google’s search results.
Why isn’t my website/webpage ranking in Google?
If you’ve carried out the above steps and confirmed that one of your webpages or your site as a whole isn’t ranking in Google, here are a few reasons why that might be the case and what you can do to fix the issue.
1) Your website/webpage doesn’t have any incoming links
Google discovers websites and pages by following links much like human users, albeit at far greater speed. That means if a website or a webpage has no incoming link, then Google won’t be able to find include the site/page in its search results.
So make sure your website’s homepage is linked to from elsewhere on the web. For example, you can submit your site to relevant business directories, or do PR to get press coverage (and hopefully links) from relevant publications.
Your website’s other pages don’t have to be linked to from other websites, but they do at least have to be linked to from pages on your own website. If your site has a logical site structure, this should happen naturally.
But sometimes over the course of a site’s lifespan, new pages can be added without reference to the original site structure.
Remember: If a page doesn’t have any incoming links, search engines can’t find it and website users can’t navigate to it from anywhere on your site.
Fortunately, if you’re running a site with just a few pages, it should be easy enough to find and fix any orphaned pages.
It’s also a good idea to create a sitemap and upload it to GSC. A sitemap will help Google understand the structure of your site. You can learn about creating a sitemap in this guide. Remember to update and resubmit your sitemap whenever you make changes to your site.
There is also a tool you can use for individual URLs. As discussed earlier, you don’t have to use this tool (and, because there’s a limit to the number of URLs you can upload, you won’t be able to use it for every page on your site anyway). It’s not certain that using the tool will get a URL added to Google any quicker, but the tool will also notify you if there are any issues with the URL. So it’s a good idea to use it for important pages as you’ll be able to identify and fix any issues straightaway.
2) You’ve accidentally blocked Google from crawling your website or page
Google needs permission to crawl a website, so if you’ve accidentally withdrawn the permission your website won’t show up in Google’s search results.
The quickest and easiest way to discover if you’ve accidentally blocked Google from accessing your website by using GSC. This short guide will help you see if you’ve blocked Google from your site.
3) There are other technical issues that are preventing Google from crawling your website or page
This shouldn’t be an issue if you’ve built your site a platform such as GoDaddy’s Website Builder, which is designed to be search-engine friendly. (Or even a CMS such as WordPress, as long as the site isn’t very complex.)
But if you’ve got a custom-built site, there’s a chance that something in the way the site was designed could hinder Google’s ability to access and understand part of, or all of, your website. (The Fetch As Google tool, part of GSC will help identify any issues.)
You can start to understand why that might be happening by reading this guide, although be warned that things do get a bit technical and you may need help implementing a solution once you’ve identified the problem.
4) Google has deliberately removed your website from its search results
If you break Google’s rules and it finds out, there’s a chance Google will remove your website from its search results completely until you have stopped breaking its rules. If this happens, you’ll be notified about it via GSC.
You can learn more about why Google might remove a site from its search results by reading its Webmaster Guidelines.
In Conclusion
I believe by now you’ll understand a little bit more about getting your pages included in search results, and why it isn’t absolutely necessary for you to submit URLs to Google.
Comments
Post a Comment