I request indexing through the Google Search Console whenever I publish a post. But sometimes Google doesn’t index it even after a few days, and I get the message “URL is not on Google”. Below, under “Page indexing”, I see the following error:
Page is not indexed: URL is unknown to Google
Here’s a screenshot:
It can be frustrating to see this message even after initiating a manual indexing request via the Google Search Console. One way to troubleshoot this is to ensure that your sitemap is up to date, contains the new URL and that Google has crawled the updated sitemap.
“URL is Not on Google” Error: Sitemap Discovered URLs
The key to understanding why Google isn’t indexing your content is to look at the sitemap’s “Discovered URLs”. Open the Google Search Console, and click “Sitemaps” on the left-hand side. You should see the list of submitted Sitemaps like this:
If you use WordPress with an SEO plugin like Yoast, this won’t be the main sitemap for your site. Click the submitted sitemap, and you should see a bunch of sub-maps. For example, I have a separate sitemap for categories, pages, posts, and tags like this:
Next to each sitemap is a field showing the last day on which Google indexed each sitemap. If your URL is not on Google, this date will be before your publication date. For example, since my content is a post, the relevant sitemap is “post-sitemap.xml”. Clicking it, I see that the number of discovered URLs is 200, as shown here:
This means that for the “post-sitemap.xml” file, Google has only crawled 200 URLs. But when I open the sitemap in my browser, I see this;
My sitemap contains 201 URLs! So that’s one URL that Google hasn’t yet discovered. Fixing this problem will make Google crawl your URL and index it.
Check if Your New URL is Available on the Sitemap
If the number of discovered URLs is the same as the number of URLs in your sitemap, scroll down the list of URLs in your sitemap to check if the new URL is showing in the list. If it doesn’t, it means you’re caching your sitemap or your sitemap tool hasn’t updated it. Here are two ways to fix it.
Fix 1: Ensure that You’re Not Caching the Sitemap
A cached sitemap means that crawlers like Google are not getting the latest information about new URLs on your site. So the first fix is ensuring your web server isn’t caching the sitemap. This can happen if you use a caching plugin like W3 Total Cache or WP Super Cache and don’t configure it properly. Even the LiteSpeed caching plugin can accidentally cache your sitemap. Or you might have enabled the option without knowing it.
Another source of caching is Cloudflare. Make sure you test all your page rules and cache rules to ensure that Cloudflare doesn’t cache your sitemap.
If you’re using an older version of Yoast, you might need to manually disable sitemap caching by adding the following code to WordPress:
add_filter('wpseo_enable_xml_sitemap_transient_caching', '__return_false');
To learn how and where to add this code, check out my tutorial on creating a WordPress plugin for custom code.
Fix 2: Manually Rebuild the Sitemap
If, for whatever reason, your sitemap doesn’t show the new URL, you can manually rebuild it. On the Yoast SEO plugin, select “General” and click the “Features” tab. Scroll down to the section called “XML sitemaps” and toggle the button from “on” to “off” and then “on” again, as shown here:
This operation will force Yoast to rebuild your sitemap and, this time, hopefully, include your URL in the list. So the next time Google crawls your sitemap, it’ll find the URL and index your page.
These solutions will fix your sitemap issue, and the dreaded “URL is not on Google” error on the search console.
Speak Your Mind